-- Logs begin at Wed 2022-10-12 11:50:18 UTC, end at Wed 2022-10-12 12:55:58 UTC. -- Oct 12 12:17:22.616341 dv-cinder-846146-8-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 12 12:17:22.623614 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 12 12:17:22 2022] *** Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: nodename: dv-cinder-846146-8-iscsi Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: machine: x86_64 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: clock source: unix Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: pcre jit disabled Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: detected number of CPU cores: 4 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: current working directory: / Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: detected binary path: /usr/bin/uwsgi-core Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: your processes number limit is 47680 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: your memory page size is 4096 bytes Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: detected max file descriptor number: 2048 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: lock engine: pthread robust mutexes Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: thunder lock: enabled Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 12 12:17:22.631136 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 12 12:17:22.666635 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: Python main interpreter initialized at 0x55f1d1ad19c0 Oct 12 12:17:22.666635 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: python threads support enabled Oct 12 12:17:22.666635 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: your server socket listen backlog is limited to 100 connections Oct 12 12:17:22.666635 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: your mercy for graceful operations on workers is 90 seconds Oct 12 12:17:22.666989 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: mapped 403077 bytes (393 KB) for 2 cores Oct 12 12:17:22.667071 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: *** Operational MODE: preforking *** Oct 12 12:17:22.667328 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: *** uWSGI is running in multiple interpreter mode *** Oct 12 12:17:22.667328 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: spawned uWSGI master process (pid: 81414) Oct 12 12:17:22.667527 dv-cinder-846146-8-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 12 12:17:22.668381 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: spawned uWSGI worker 1 (pid: 81424, cores: 1) Oct 12 12:17:22.668381 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: spawned uWSGI worker 2 (pid: 81425, cores: 1) Oct 12 12:17:22.668537 dv-cinder-846146-8-iscsi devstack@keystone.service[81414]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 12 12:17:26.943031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:17:26.943031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: from cryptography.utils import int_from_bytes Oct 12 12:17:26.943822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:17:26.943822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: from cryptography.utils import int_from_bytes Oct 12 12:17:26.945089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:17:26.945089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: from cryptography.utils import int_from_bytes Oct 12 12:17:26.945593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 12 12:17:26.945593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: from cryptography.utils import int_from_bytes Oct 12 12:17:26.993122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 12:17:26.993296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 12:17:26.993513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] command line args: [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 12:17:26.993783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 12:17:26.993958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ================================================================================ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 12:17:26.994208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] admin_token = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.994450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_dir = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.994635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.994813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] config_source = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.994991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] debug = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.995123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 12 12:17:26.995205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.995447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 12 12:17:26.995513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] command line args: [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 12 12:17:26.995570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.995712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.995804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 12 12:17:26.995920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ================================================================================ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 12 12:17:26.996018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] admin_token = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_dir = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_config_append = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.996919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] config_source = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_dir = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] debug = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.997963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_config_append = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_dir = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.998900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:26.999930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.000998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] publish_errors = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.001936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] standard_threads = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.002909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.003065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.003199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] publish_errors = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.003291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.003370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.003447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_journal = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.003518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_json = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_stderr = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] use_syslog = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.004890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.005032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.005107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.005274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] standard_threads = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.005418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.005536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.005615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.005783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.005869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.005961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.006039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.006138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_journal = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.006287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.006383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_json = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.006457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_stderr = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.006526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.006633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] use_syslog = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.006755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.006866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 12 12:17:27.007002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.007939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008343 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.008904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.009888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.010975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.011933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.012936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.013895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.014947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.015956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.016874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.017789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.018967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.019088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.020918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.021934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.022900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.023983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.024927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.025973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.026916 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.027980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.028981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.029946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.external = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.030867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.password = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] auth.token = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.031946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.032982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.external = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.033943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.password = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] auth.token = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.034951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.035930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.036936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.037957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.038993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.039885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.040918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.041984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.042944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043584 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.043973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.044989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.045945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.046957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.047993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.048892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.049936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.050067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.050192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.050319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.050642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.050792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.050930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.051948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.052963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.053992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.054947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.055979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.056989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.057953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058692 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.058989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.059943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.060980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.061993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.062944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.063897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.064927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.065887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.066543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.066638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.066798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.066922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.067929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.068906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.069875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.070987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.071951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.072921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.073876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.074978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.075923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.076959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.077909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.078849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.079953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.080042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.080185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.080266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.080339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81424) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 12:17:27.080435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.080676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.080911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.081146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.081403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.081691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.082598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.083025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.083282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.083526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.083779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.084019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.084265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.084495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.084734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.084970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 12 12:17:27.085140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81425) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 12 12:17:27.110061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.110285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.110503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.113263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.113709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.113990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.118926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.122225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.148150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.148367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.149608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.149842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.201337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.201804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.249888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.250112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.251379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.251611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.407162 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.407399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.407618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.407832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.407993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.408103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.408227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.408313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.408451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.408670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.408885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.409098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.431454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.431680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.431768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.431903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.431990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.432978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.443876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.444469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.456436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.457120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.457387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.457504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.457614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.457820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.457916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.458024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.458161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.458248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81424) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.458385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.458608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.458829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.459046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81425) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 12 12:17:27.475488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.475694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:17:27.475780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.475950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:17:27.476033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.476216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:17:27.476300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.476468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 12 12:17:27.476551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.476737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.476827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.476994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.477079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.477248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.477349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.477532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.486821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.487949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.494760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.495272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.497672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.497958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.498161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.498246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.498445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.498530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.498729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.498814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.499284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.499375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.499550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.499641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.499811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.499892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.500889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.501157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.523897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.524908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.525373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.525660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.526390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.526685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.530313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.530599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.530877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.531314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.531599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.531881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.537829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.538113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.539162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.539448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.543666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.543960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.544250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.544539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.544818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.545007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.545093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.545330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.545415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.545632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.545777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.545900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.546172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.546445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.546713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.546980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.561022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.561299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.562681 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.562968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.565160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.567183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.568527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.568822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.570428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.570730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.573024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.574930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.576235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.578077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.579452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.579749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.580033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.580315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.580593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.580881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.581275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.582108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.582606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.583036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.583323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.583618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.598095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.598471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.598740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.599005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.600270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.600558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.600826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.601093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.605854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.607382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.607922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.608887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.609164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.609476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.611002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.611284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.611815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.612100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.612385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.612664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.613642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.614797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.615082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.615384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.615665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.616054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.624959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.625241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.625545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.625828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.626207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.626511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.626791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.627451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.627735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.628016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.628305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.629368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.629817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.630104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.642691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.645758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.645892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.648605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.648691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.650231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.651538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.651721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.652003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.652411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.652969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.654591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.654876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.655283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.658758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.660746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.662760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.663643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.664039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.664229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.664313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.664588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.667105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.667496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.667783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.668063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.672385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.672677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.672958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.676690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.677083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.677390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.680442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81424) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.680729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.681009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.681282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.681588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.681860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.682135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.682419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.682691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.682975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.683248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.683520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.683783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.684042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81424) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.684244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81425) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 12 12:17:27.684530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.684812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.685089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.685387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.685661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.685934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.686226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.686495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.686768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.687036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.687303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.687566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.687836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81425) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 12 12:17:27.718872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f1d1ad19c0 pid: 81424 (default app) Oct 12 12:17:27.722253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9eb2d-1342-447d-a49a-72876b380fb2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:27.722566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9eb2d-1342-447d-a49a-72876b380fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:27.722906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9eb2d-1342-447d-a49a-72876b380fb2 None None] PATH_INFO: `/v3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:27.723653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f1d1ad19c0 pid: 81425 (default app) Oct 12 12:17:27.726951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Wed Oct 12 12:17:27 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:34.462935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:34.463434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:34.463541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:34.467689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1/2] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:34.473184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:34.473659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:34.473800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:34.543231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: INFO dbcounter [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] Registered counter for database keystone Oct 12 12:17:34.561554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG oslo_db.sqlalchemy.engines [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81424) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 12:17:34.568097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writer thread running {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 12:17:34.742613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG passlib.handlers.bcrypt [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81424) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 12:17:34.742988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG passlib.handlers.bcrypt [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81424) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 12:17:34.801828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:34.803352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:34.932293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 2/3] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:34 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 460 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:17:35.395763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:35.465392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: INFO dbcounter [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] Registered counter for database keystone Oct 12 12:17:35.486569 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG oslo_db.sqlalchemy.engines [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81425) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 12 12:17:35.493240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writer thread running {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 12 12:17:35.618745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fb72f-2313-448d-8dc9-e29c16e3b9d5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:35.622896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:35.623792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:35.624099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:35.624395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:35.632552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:35.684286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 12:17:35.684631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76072a-086d-451c-aa99-03e6db754ed2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:35.693906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 2/4] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Oct 12 12:17:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:17:35.699839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:35.716966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c84937-e852-4bdc-b0b1-bd69ee1147b7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:35.721333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:35.723111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:35.723384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:35.723653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] PATH_INFO: `/v3/projects/22af725251c64418b4185f46ec322cf9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:35.725219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] RBAC: Authorizing `identity:get_project(project_id=22af725251c64418b4185f46ec322cf9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:35.777492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING keystone.common.rbac_enforcer.enforcer [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 12 12:17:35.777847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771128a0-96a6-4b87-9f9e-c19125ea53f8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:35.782326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 3/5] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Oct 12 12:17:35 2022] GET /identity/v3/projects/22af725251c64418b4185f46ec322cf9 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:17:37.901769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902a89-deb9-4730-8aa5-9f3d7945c475 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:37.902303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902a89-deb9-4730-8aa5-9f3d7945c475 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:37.902303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902a89-deb9-4730-8aa5-9f3d7945c475 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:37.906448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 3/6] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:17:37.912189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:37.912587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:37.912750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:37.933559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:37.935089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:37.943588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 4/7] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 12:17:38.208532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902a89-deb9-4730-8aa5-9f3d7945c475 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:38.220481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902a89-deb9-4730-8aa5-9f3d7945c475 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:38.225201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3b3aa-e2e0-4f45-a8e3-c9ec8295c9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:38.225823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b3aa-e2e0-4f45-a8e3-c9ec8295c9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:38.226106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b3aa-e2e0-4f45-a8e3-c9ec8295c9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:38.226380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b3aa-e2e0-4f45-a8e3-c9ec8295c9f7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:38.233925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3b3aa-e2e0-4f45-a8e3-c9ec8295c9f7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:38.235713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3b3aa-e2e0-4f45-a8e3-c9ec8295c9f7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:38.245134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 4/8] 10.222.0.17 () {58 vars in 1324 bytes} [Wed Oct 12 12:17:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:17:38.250802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:38.260105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ce99a-6b8a-4ad9-bfbc-2fd1e4289b5f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:38.264480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b517dd-de5c-4299-a640-4ec66d7ef6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:38.265103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b517dd-de5c-4299-a640-4ec66d7ef6bb admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:38.265410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b517dd-de5c-4299-a640-4ec66d7ef6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:38.265678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b517dd-de5c-4299-a640-4ec66d7ef6bb admin admin] PATH_INFO: `/v3/users/8feb61a946fd40a291adfe0eb3820253` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:38.266977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b517dd-de5c-4299-a640-4ec66d7ef6bb admin admin] RBAC: Authorizing `identity:get_user(user_id=8feb61a946fd40a291adfe0eb3820253)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:38.268939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b517dd-de5c-4299-a640-4ec66d7ef6bb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:38.270566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 5/9] 10.222.0.17 () {56 vars in 1198 bytes} [Wed Oct 12 12:17:38 2022] GET /identity/v3/users/8feb61a946fd40a291adfe0eb3820253 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:40.445621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57565-652e-40f1-bc1c-89c0ed5f7edb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:40.446141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57565-652e-40f1-bc1c-89c0ed5f7edb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:40.446254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d57565-652e-40f1-bc1c-89c0ed5f7edb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:40.447477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 5/10] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:40.452665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:40.453058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:40.453206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:40.471476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:40.472647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:40.517300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3ffc5-9471-4cb2-bf1f-d930fb3582e3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:40.517535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3ffc5-9471-4cb2-bf1f-d930fb3582e3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:40.517813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3ffc5-9471-4cb2-bf1f-d930fb3582e3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:40.519870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 6/11] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:17:40.524936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:40.525218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:40.525522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:40.553456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 6/12] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:40.567784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG passlib.handlers.bcrypt [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81425) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 12 12:17:40.568177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG passlib.handlers.bcrypt [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81425) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 12 12:17:40.616956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:40.618589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:40.717659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 7/13] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:17:40.819677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:40.844752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:40.926367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2862ab2-d9b5-43f9-b328-08e9236d990e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:40.959938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:40.960558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:40.960833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:40.961107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:40.973021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:17:40.981737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:40.982617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:40.983435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 7/14] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:17:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:17:40.988952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:40.993290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:41.074004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:41.089995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540cd52-19e4-4516-9044-07f873d2dc57 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.092372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:41.093143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9add174-7150-4acc-be0e-90b8a7f6fcb4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.129014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.129203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.129642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.129916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.130183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.130277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.130609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.130845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.130989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.131889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:41.132191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.132285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.133648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.141704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.sql.core [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:17:41.141704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:17:41.141704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [parameters: {'id': '4b9fd8fb852842af8bd68d6bdcf3294d', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:17:41.141704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:17:41.145436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 8/15] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:17:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:41.150865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:17:41.159603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 12:17:41.160800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application yield resource Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise self.value Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application session.commit() Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:17:41.161953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:17:41.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:17:41.164046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:17:41.164963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [parameters: {'id': '4b9fd8fb852842af8bd68d6bdcf3294d', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 12:17:41.166026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:41.166929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.168045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 8/16] 10.222.0.17 () {60 vars in 1137 bytes} [Wed Oct 12 12:17:40 2022] POST /identity/v3/domains => generated 190 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:17:41.174512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:41.261578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbd956-008d-4a2e-a477-e1827e2d5f81 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.275504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e3519e-7b2e-4ed2-a26c-bbef2010150f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.295908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.296757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.297031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.297299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] PATH_INFO: `/v3/users/8feb61a946fd40a291adfe0eb3820253` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.298804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] RBAC: Authorizing `identity:get_user(user_id=8feb61a946fd40a291adfe0eb3820253)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.301219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.302819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 9/17] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:17:41 2022] GET /identity/v3/users/8feb61a946fd40a291adfe0eb3820253 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:17:41.308100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:41.312095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.312720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.312998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.313268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.323357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.324859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.332021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:17:41.341607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:41.342671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:17:41.343627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:17:41.343627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:17:41.343627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:41.343851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:41.343934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 9/18] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:17:41.349755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:41.408219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a538-5a9e-4d7c-b2c6-948ee4ab0c9b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.441976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.442689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.443038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.443378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.444574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.446159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.447432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4e991a-62fa-4dfc-95f0-8e53deee9fc3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.448144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 10/19] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:17:41.453840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:41.480952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.481592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.481874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.482148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.482792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:41.483095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.484474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d9dfc-579d-47ad-80c2-4bcd5dfb9a0f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.501885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:17:41.556588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c0971-4ce7-4bb8-80aa-4d31cf3815a9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:41.558454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:41.593227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90ff87-c57e-4de2-a644-de159aa307f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:41.593851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90ff87-c57e-4de2-a644-de159aa307f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:41.594133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90ff87-c57e-4de2-a644-de159aa307f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:41.594412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90ff87-c57e-4de2-a644-de159aa307f8 None admin] PATH_INFO: `/v3/domains/default/users/8feb61a946fd40a291adfe0eb3820253/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:41.596453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90ff87-c57e-4de2-a644-de159aa307f8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8feb61a946fd40a291adfe0eb3820253, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:41.597986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90ff87-c57e-4de2-a644-de159aa307f8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:41.717635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 11/21] 10.222.0.17 () {58 vars in 1361 bytes} [Wed Oct 12 12:17:41 2022] PUT /identity/v3/domains/default/users/8feb61a946fd40a291adfe0eb3820253/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:17:44.055489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bf57-e531-4ea3-af20-e9f537d213cf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.056010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bf57-e531-4ea3-af20-e9f537d213cf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.056114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a40bf57-e531-4ea3-af20-e9f537d213cf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.057642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:44.063887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.064349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.064421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.093865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:44.095390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:44.159688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2d3e7-ff2c-44f8-bd7b-617c688d4c26 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.159901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2d3e7-ff2c-44f8-bd7b-617c688d4c26 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.160115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e2d3e7-ff2c-44f8-bd7b-617c688d4c26 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.161594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 12/23] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:17:44.177997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40fd9e-d926-4705-bba9-3575a84923e5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.178288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40fd9e-d926-4705-bba9-3575a84923e5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.178550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba40fd9e-d926-4705-bba9-3575a84923e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.181697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 12/24] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:17:44.214319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ba40fd9e-d926-4705-bba9-3575a84923e5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:44.215533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ba40fd9e-d926-4705-bba9-3575a84923e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:44.301006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:44.319667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 13/25] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:44.542318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:44.661065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:44.671043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a2d0af-f9e7-44eb-84ee-458c3b22357e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:44.682964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907c9fe-68e2-4eea-b9e9-30329e61688d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.683290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907c9fe-68e2-4eea-b9e9-30329e61688d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.683509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2907c9fe-68e2-4eea-b9e9-30329e61688d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.685153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 14/26] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:44.690975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.691276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.691615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.728329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:44.729551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:44.764886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:44.765553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:44.765832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:44.766104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:44.804052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:44.805797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:17:44.843689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:44.844597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:17:44.845461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:17:44.845461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:17:44.845461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:17:44.850227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 13/27] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:17:44.851074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:44.885685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 15/28] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:17:44.886532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:44.985438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546a6e58-400a-467d-8dc7-1c388a370908 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.022657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a6a762-b985-4cd2-975c-466864b8b067 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.024261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03036d58-0d6a-406d-b367-9245f7ad60e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.025882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03036d58-0d6a-406d-b367-9245f7ad60e1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.026311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03036d58-0d6a-406d-b367-9245f7ad60e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.026783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03036d58-0d6a-406d-b367-9245f7ad60e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.050432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03036d58-0d6a-406d-b367-9245f7ad60e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-03036d58-0d6a-406d-b367-9245f7ad60e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:17:45.057361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:45.058267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:17:45.059007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:17:45.061686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 14/29] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:17:45.063081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e3fcf-dc56-4d02-8f72-1916b2b9a92c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.063583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e3fcf-dc56-4d02-8f72-1916b2b9a92c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.064056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e3fcf-dc56-4d02-8f72-1916b2b9a92c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.065656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:45.070097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 15/30] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:45.071054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e3fcf-dc56-4d02-8f72-1916b2b9a92c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:45.083095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:45.096709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.098454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.099080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.100660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.102054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:45.102842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.107120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d1470-9218-4c49-9a51-e344fff41448 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.126190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:45.128911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.130716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.131373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.159922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:45.161472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:45.233905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 17/32] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:45.235302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:45.325178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535e3fcf-dc56-4d02-8f72-1916b2b9a92c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.360708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40a394b-21e9-4c83-8ad9-0610c5723ad7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.373867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.374495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.374773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.375046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.375718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:45.376025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.378155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.381646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:45.395142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 16/33] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:17:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:45.395977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:45.432772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.433415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.433715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.433971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.434633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.436098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.523913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:45.566909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0992de-436c-47ab-8aa2-83faed2f583e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.601698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.602348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.602632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.602902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.603570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.604935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.624019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 17/34] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Oct 12 12:17:45 2022] POST /identity/v3/roles => generated 225 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:17:45.625071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:45.628472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 18/35] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:17:45 2022] POST /identity/v3/projects => generated 301 bytes in 393 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:17:45.629610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:45.756012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45337bb-aa9f-4ac5-b673-adbe295832d8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.787780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:45.809346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.809346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.809346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.809346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] PATH_INFO: `/v3/users/8feb61a946fd40a291adfe0eb3820253` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.809745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] RBAC: Authorizing `identity:get_user(user_id=8feb61a946fd40a291adfe0eb3820253)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.809745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.810972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 18/36] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:17:45 2022] GET /identity/v3/users/8feb61a946fd40a291adfe0eb3820253 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:45.816842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:45.856908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ee1ee5-3d91-413d-bb09-565611d496ab None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.898809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8063e1-547f-4da7-98f1-167d25ad4fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.899794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8063e1-547f-4da7-98f1-167d25ad4fb9 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.900274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8063e1-547f-4da7-98f1-167d25ad4fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.901685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8063e1-547f-4da7-98f1-167d25ad4fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.902645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8063e1-547f-4da7-98f1-167d25ad4fb9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.904462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8063e1-547f-4da7-98f1-167d25ad4fb9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.930075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 19/37] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Oct 12 12:17:45 2022] POST /identity/v3/roles => generated 219 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:17:45.932624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553822da-1796-4ee1-8db0-752016bad4c3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.932962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553822da-1796-4ee1-8db0-752016bad4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.933185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553822da-1796-4ee1-8db0-752016bad4c3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.934776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 20/38] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:45.936338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdf9881-f529-41dc-9404-d01613eb7f07 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:45.941709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de57b1-3a7e-4731-8143-59e02d8b4535 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.942103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de57b1-3a7e-4731-8143-59e02d8b4535 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.942341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de57b1-3a7e-4731-8143-59e02d8b4535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.970910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d3de57b1-3a7e-4731-8143-59e02d8b4535 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:45.973194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d3de57b1-3a7e-4731-8143-59e02d8b4535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:45.989651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:45.990595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:45.991074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:45.991524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:45.993417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:45.996281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:45.998949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 19/39] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:17:46.006121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:46.018813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:46.099596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:46.147472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d6543-a08c-46bd-891f-8b6386eed85d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:46.191384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:46.191384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:46.191384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:46.191384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:46.193116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.domain.id=default, user.id=8feb61a946fd40a291adfe0eb3820253) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:46.193116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:46.194521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:46.197943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 21/40] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:46.199218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa8fa-b9a3-4aba-81f8-fff3bf73c27e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:46.200004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa8fa-b9a3-4aba-81f8-fff3bf73c27e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:46.200004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aa8fa-b9a3-4aba-81f8-fff3bf73c27e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:46.201011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 22/41] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:46.203386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 20/42] 10.222.0.17 () {56 vars in 1322 bytes} [Wed Oct 12 12:17:46 2022] GET /identity/v3/role_assignments?user.id=8feb61a946fd40a291adfe0eb3820253&scope.domain.id=default&role.id=2e857246137f4d06b40cdabedf42881f => generated 530 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:17:46.207501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:46.207788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:46.208060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:46.228465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:46.229736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:46.308229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 23/43] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:46.472530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:46.570785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7d50c4-e004-4e5d-8602-cb83a87983fc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:46.587197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:46.594982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:46.604311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:46.604941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:46.605221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:46.605520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:46.606614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:46.608068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:46.609646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 21/44] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:46.615625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:46.667760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:46.688101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d6641-e8de-4c78-b02d-c994bd0aee00 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:46.713085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef810d-cd25-4912-a28e-a250a2ab310c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:46.721641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d884e6-3544-40bb-8d13-3d7c188546b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:46.722255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d884e6-3544-40bb-8d13-3d7c188546b9 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:46.722589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d884e6-3544-40bb-8d13-3d7c188546b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:46.722811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d884e6-3544-40bb-8d13-3d7c188546b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:46.723473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d884e6-3544-40bb-8d13-3d7c188546b9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:46.724964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d884e6-3544-40bb-8d13-3d7c188546b9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:46.740655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 24/45] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Oct 12 12:17:46 2022] POST /identity/v3/roles => generated 223 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:17:46.748825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a01cb61-df7a-432a-b9ce-b1fe78c4f4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:46.749563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01cb61-df7a-432a-b9ce-b1fe78c4f4d8 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:46.749850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01cb61-df7a-432a-b9ce-b1fe78c4f4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:46.750130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a01cb61-df7a-432a-b9ce-b1fe78c4f4d8 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:46.750806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01cb61-df7a-432a-b9ce-b1fe78c4f4d8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:46.752465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a01cb61-df7a-432a-b9ce-b1fe78c4f4d8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:46.784641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 22/46] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:17:46 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:17:48.952623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3098d-de09-487b-8b02-f00ee7320346 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:48.953146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3098d-de09-487b-8b02-f00ee7320346 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:48.953146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a3098d-de09-487b-8b02-f00ee7320346 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:48.954539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 25/47] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:48.960092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:48.960486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:48.960659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:48.979163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:48.980384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:49.023269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:49.049506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 23/48] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:49.319309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3098d-de09-487b-8b02-f00ee7320346 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:49.418770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a3098d-de09-487b-8b02-f00ee7320346 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:49.451451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e912b9-d227-497b-800c-90c96ed888df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:49.452070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e912b9-d227-497b-800c-90c96ed888df None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:49.452360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e912b9-d227-497b-800c-90c96ed888df None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:49.452634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e912b9-d227-497b-800c-90c96ed888df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:49.453771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e912b9-d227-497b-800c-90c96ed888df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:49.455797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e912b9-d227-497b-800c-90c96ed888df None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:49.457349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 26/49] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:49.462963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:49.556236 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec5535-68a5-486a-a197-1320a88aca5b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:49.588836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9170566-3565-41cb-910c-6916904b25b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:49.589453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9170566-3565-41cb-910c-6916904b25b1 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:49.589734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9170566-3565-41cb-910c-6916904b25b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:49.590008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9170566-3565-41cb-910c-6916904b25b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:49.590646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9170566-3565-41cb-910c-6916904b25b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:49.592096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9170566-3565-41cb-910c-6916904b25b1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:49.617270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 24/50] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Oct 12 12:17:49 2022] POST /identity/v3/projects => generated 298 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:17:51.775415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c230a-f3b9-460e-af44-cfb78416db5f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:51.775941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c230a-f3b9-460e-af44-cfb78416db5f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:51.776022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c230a-f3b9-460e-af44-cfb78416db5f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:51.777372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 27/51] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:51.782591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:51.782982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:51.783174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:51.801000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:51.802229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:51.824770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:51.868617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 25/52] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:52.137158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c230a-f3b9-460e-af44-cfb78416db5f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:52.148676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:52.239868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c230a-f3b9-460e-af44-cfb78416db5f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:52.273561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e92560-56f0-47dc-b9ec-5444c8e3d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:52.274192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e92560-56f0-47dc-b9ec-5444c8e3d7aa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:52.274475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e92560-56f0-47dc-b9ec-5444c8e3d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:52.274751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e92560-56f0-47dc-b9ec-5444c8e3d7aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:52.275867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e92560-56f0-47dc-b9ec-5444c8e3d7aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:52.277353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e92560-56f0-47dc-b9ec-5444c8e3d7aa None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:52.278910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:52.284461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:52.381249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06ad4a-4197-46ee-a6b6-f73aab6519cc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:52.415377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c6e73-19a4-4ee5-9d94-bb937a4453d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:52.415917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c6e73-19a4-4ee5-9d94-bb937a4453d9 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:52.416200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c6e73-19a4-4ee5-9d94-bb937a4453d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:52.416477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c6e73-19a4-4ee5-9d94-bb937a4453d9 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:52.417129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c6e73-19a4-4ee5-9d94-bb937a4453d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:52.418616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c6e73-19a4-4ee5-9d94-bb937a4453d9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:52.446392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 26/54] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:17:52 2022] POST /identity/v3/users => generated 275 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:17:55.618161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55737a9-4b68-4487-a2aa-7b5b63c73351 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:55.618977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55737a9-4b68-4487-a2aa-7b5b63c73351 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:55.619414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55737a9-4b68-4487-a2aa-7b5b63c73351 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:55.621183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 29/55] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:55.626553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037bb77-c495-4f09-a89d-a30a39cfbcd2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:55.627817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037bb77-c495-4f09-a89d-a30a39cfbcd2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:55.628337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6037bb77-c495-4f09-a89d-a30a39cfbcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:55.770195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6037bb77-c495-4f09-a89d-a30a39cfbcd2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:55.771424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6037bb77-c495-4f09-a89d-a30a39cfbcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:55.789946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:55.867893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 27/56] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:56.090832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983c57-afb3-4568-9171-defafd9f0c32 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.091502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983c57-afb3-4568-9171-defafd9f0c32 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.091977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24983c57-afb3-4568-9171-defafd9f0c32 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.093524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 30/57] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:56.098769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7302ba-165b-4b30-8a82-d3e22ea66281 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.099066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7302ba-165b-4b30-8a82-d3e22ea66281 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.099349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7302ba-165b-4b30-8a82-d3e22ea66281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.104887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958e150-23dd-4284-aa60-4aeeab68074b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.105429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958e150-23dd-4284-aa60-4aeeab68074b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.105899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2958e150-23dd-4284-aa60-4aeeab68074b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.107623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 31/58] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:17:56.113039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.113585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.114055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.122924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-da7302ba-165b-4b30-8a82-d3e22ea66281 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:56.124423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da7302ba-165b-4b30-8a82-d3e22ea66281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:56.140575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:56.141846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:56.210216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 28/59] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:17:56.219030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a6596-6288-46f9-8583-f4d48b1ff0b5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.219470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a6596-6288-46f9-8583-f4d48b1ff0b5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.219921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a6596-6288-46f9-8583-f4d48b1ff0b5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.222268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 29/60] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:56.228089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c498705-f08f-456a-ac1f-26d68ba41b94 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.228625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c498705-f08f-456a-ac1f-26d68ba41b94 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.229054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c498705-f08f-456a-ac1f-26d68ba41b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.252769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0c498705-f08f-456a-ac1f-26d68ba41b94 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:56.255336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0c498705-f08f-456a-ac1f-26d68ba41b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:56.271721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 32/61] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:17:56.278252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.288400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:56.403356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 30/62] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:17:56.405113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350bdae-c19d-4a04-a5f5-7fe37549d378 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.405562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350bdae-c19d-4a04-a5f5-7fe37549d378 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.405972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c350bdae-c19d-4a04-a5f5-7fe37549d378 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.413028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 31/63] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:17:56.415104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.415528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.415940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.426238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:56.444514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf9c810-03b3-4498-bf75-86602993d675 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:56.447716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:17:56.448926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:17:56.523227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:56.524503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.525237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.526179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.537000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 32/64] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:17:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:17:56.538115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.540685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:17:56.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:56.548446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:17:56.549216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:56.549841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 33/65] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:17:56.554913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.640794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:56.673349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0652c0c-21de-4d4b-b541-f84f5936f8cd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:56.691118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da5208-7431-4620-9287-fb64e386d0af None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:56.716767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:56.717663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.718085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.718517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.720990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:56.731219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:56.731410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:17:56.732036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.732413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.732785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.733643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:56.734042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:56.735533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:17:56.738480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:56.739432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:17:56.740185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:17:56.742423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 33/66] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:17:56.749197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.751377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 34/67] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:56.753775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.857287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862842f-3d99-4899-a47a-bfc8703b392c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:56.859758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a94a-053e-4b83-8f6d-9389c1e7cc39 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:56.885522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:56.891237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:56.891875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.892716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.893021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.894400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:56.895059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:56.895343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:56.895615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:56.903125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:17:56.905839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:17:56.908557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:56.909418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:17:56.910182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 34/68] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:17:56.910800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:17:56.911606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:17:56.912431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:17:56.913223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 35/69] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:17:56.916044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:56.966607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.012534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19692dd-a6a1-49c3-a261-27f1d722cf0b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.013692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c794f3-16d7-4334-a645-38c0a8d9bdf6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.046076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.046281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.046708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.046880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.046994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.047166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.047274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] PATH_INFO: `/v3/users/c0edd4c51fd342338cf2652e2ee1df11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.047440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.048099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:57.048476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.050338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.058187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 35/70] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.058976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.062844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] RBAC: Authorizing `identity:get_user(user_id=c0edd4c51fd342338cf2652e2ee1df11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.064956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.066642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 36/71] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:17:56 2022] GET /identity/v3/users/c0edd4c51fd342338cf2652e2ee1df11 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.071042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.112024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.162848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de314cd-d1c9-4e9a-83c9-731b14d6046a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.164925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd85196-fde5-4911-bac5-5d98bc458239 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.178879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.197927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.198555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.198843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.199119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.199214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.199802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.200142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.200302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.200448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.201112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:57.201452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.201715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.202808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.203290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 36/72] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.204060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.212286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 37/73] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.214161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.310039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2fb74-0aa5-4736-a564-23cb6a4e5175 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.311596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10444d9e-c8c1-4ecf-ae7e-8140b85ed2df None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.320743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.343899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.344505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.344789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.344902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.345064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.345589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.345767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:17:57.346051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.346142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.346337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] PATH_INFO: `/v3/users/c0edd4c51fd342338cf2652e2ee1df11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.347441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.347535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] RBAC: Authorizing `identity:get_user(user_id=c0edd4c51fd342338cf2652e2ee1df11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.348927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.350705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 38/74] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/users/c0edd4c51fd342338cf2652e2ee1df11 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.351300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.356096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 37/75] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.356764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.405839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.451413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f86470-4477-4d96-9156-48953b7e74af None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.460100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63f5ed-c2bf-473d-9291-dc850fadc69b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.486964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.487598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.487885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.488164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.489328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.490803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.492465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 39/76] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.492934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.493224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.493564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.493848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.494123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.494773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.496221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.520202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 38/77] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:17:57 2022] POST /identity/v3/users => generated 289 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:17:57.520944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.545194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.612925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec7965-dce2-4fcb-8906-fc5c4658f339 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.650440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a533fdc6-b10a-41e4-aaf9-a2922af3b5b8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.652191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.656161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.656873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.657119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.657403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] PATH_INFO: `/v3/users/c0edd4c51fd342338cf2652e2ee1df11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.659573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] RBAC: Authorizing `identity:get_user(user_id=c0edd4c51fd342338cf2652e2ee1df11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.661096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.662953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 40/78] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/users/c0edd4c51fd342338cf2652e2ee1df11 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.668340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.690162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.690817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.691091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.691367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.692578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.694113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.695750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 39/79] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.700471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.789326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19874337-e87a-4798-822a-50d5f8ee8811 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.804250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.825070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e332abec-7433-47ea-a466-76123057cb41 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:57.830702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.831356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.831640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.831922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] PATH_INFO: `/v3/users/8feb61a946fd40a291adfe0eb3820253` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.833097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] RBAC: Authorizing `identity:get_user(user_id=8feb61a946fd40a291adfe0eb3820253)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.835143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.837342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 41/80] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/users/8feb61a946fd40a291adfe0eb3820253 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:57.838399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.882365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:57.883039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:57.883327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:57.883605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5/users/c0edd4c51fd342338cf2652e2ee1df11/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:57.889187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=43e403631cd6479489b75c17c38466d5, user_id=c0edd4c51fd342338cf2652e2ee1df11, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:57.890680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:57.906092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 40/81] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:17:57 2022] PUT /identity/v3/projects/43e403631cd6479489b75c17c38466d5/users/c0edd4c51fd342338cf2652e2ee1df11/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:17:57.907358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:57.956133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:57.990952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0680494-523e-431a-900c-3fb6535e495f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:58.040094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d71c60-46c2-4b76-8ac4-b91b780391a0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:58.087888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:58.088568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:58.088849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:58.089121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5/users/c0edd4c51fd342338cf2652e2ee1df11/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:58.091620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=43e403631cd6479489b75c17c38466d5, user_id=c0edd4c51fd342338cf2652e2ee1df11, role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:58.093373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:58.118270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 41/82] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:17:57 2022] PUT /identity/v3/projects/43e403631cd6479489b75c17c38466d5/users/c0edd4c51fd342338cf2652e2ee1df11/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:17:58.118629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:58.119411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:58.120246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:58.120575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:58.120862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] PATH_INFO: `/v3/projects/6db4fff2382a4e01ab2e98ba9116f1be` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:58.124209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] RBAC: Authorizing `identity:get_project(project_id=6db4fff2382a4e01ab2e98ba9116f1be)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:58.129597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:58.138472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 42/83] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:17:57 2022] GET /identity/v3/projects/6db4fff2382a4e01ab2e98ba9116f1be => generated 312 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:58.145064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:58.256024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:58.273999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:17:58.282642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda622-ee46-4b37-9b16-6eea1eba90f1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:58.324429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:58.325017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:58.325288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:58.325602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:58.326842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:58.328617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:58.331581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 42/84] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:17:58 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:17:58.337475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:17:58.406513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a282506-341b-4da0-ba57-47d8c89d8818 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:58.454333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c38e46-0581-40bc-9739-363b96bd928e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:17:58.495535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c9b862-2dd9-4e47-bd7d-9a198dcb7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:58.496286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9b862-2dd9-4e47-bd7d-9a198dcb7075 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:58.496572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9b862-2dd9-4e47-bd7d-9a198dcb7075 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:58.496853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c9b862-2dd9-4e47-bd7d-9a198dcb7075 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5/users/8feb61a946fd40a291adfe0eb3820253/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:58.499112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9b862-2dd9-4e47-bd7d-9a198dcb7075 None admin] RBAC: Authorizing `identity:create_grant(project_id=43e403631cd6479489b75c17c38466d5, user_id=8feb61a946fd40a291adfe0eb3820253, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:58.500590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c9b862-2dd9-4e47-bd7d-9a198dcb7075 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:58.515290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5cd9e-4c23-470a-8987-2b3cab4f53ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:17:58.516014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 43/85] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:17:58 2022] PUT /identity/v3/projects/43e403631cd6479489b75c17c38466d5/users/8feb61a946fd40a291adfe0eb3820253/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:17:58.516130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5cd9e-4c23-470a-8987-2b3cab4f53ad None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:17:58.516341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5cd9e-4c23-470a-8987-2b3cab4f53ad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:17:58.516597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5cd9e-4c23-470a-8987-2b3cab4f53ad None admin] PATH_INFO: `/v3/projects/6db4fff2382a4e01ab2e98ba9116f1be/users/c0edd4c51fd342338cf2652e2ee1df11/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:17:58.519424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5cd9e-4c23-470a-8987-2b3cab4f53ad None admin] RBAC: Authorizing `identity:create_grant(project_id=6db4fff2382a4e01ab2e98ba9116f1be, user_id=c0edd4c51fd342338cf2652e2ee1df11, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:17:58.521427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5cd9e-4c23-470a-8987-2b3cab4f53ad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:17:58.540861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 43/86] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:17:58 2022] PUT /identity/v3/projects/6db4fff2382a4e01ab2e98ba9116f1be/users/c0edd4c51fd342338cf2652e2ee1df11/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:00.253696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc8cd5-edfd-4adf-8a8a-03aac2e78c57 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:00.254210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc8cd5-edfd-4adf-8a8a-03aac2e78c57 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:00.254282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbc8cd5-edfd-4adf-8a8a-03aac2e78c57 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:00.255839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 44/87] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:00.271622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532930b1-91bf-4023-a107-071cb5f6a819 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:00.272060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532930b1-91bf-4023-a107-071cb5f6a819 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:00.272233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532930b1-91bf-4023-a107-071cb5f6a819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:00.302359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-532930b1-91bf-4023-a107-071cb5f6a819 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:00.303593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-532930b1-91bf-4023-a107-071cb5f6a819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:00.406505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 44/88] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:18:00.561662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbc24-2408-4146-bc14-c0dc67278a74 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:00.561980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbc24-2408-4146-bc14-c0dc67278a74 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:00.562149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5fbc24-2408-4146-bc14-c0dc67278a74 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:00.563494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 45/89] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:00.569056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd538f-cacb-4cb1-a9a8-f25ba185c9bf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:00.569371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd538f-cacb-4cb1-a9a8-f25ba185c9bf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:00.569673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd538f-cacb-4cb1-a9a8-f25ba185c9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:00.594329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-21cd538f-cacb-4cb1-a9a8-f25ba185c9bf None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:00.596376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21cd538f-cacb-4cb1-a9a8-f25ba185c9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:00.636549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:00.679790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 45/90] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:00.874472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbc24-2408-4146-bc14-c0dc67278a74 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:00.895582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:00.951774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014fe9d-c9ea-46e0-8627-b66410e67c0c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:00.952071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014fe9d-c9ea-46e0-8627-b66410e67c0c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:00.952346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d014fe9d-c9ea-46e0-8627-b66410e67c0c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:00.954159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 46/91] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:00.967119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:00.967453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:00.967681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.000513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:01.002069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:01.025152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5fbc24-2408-4146-bc14-c0dc67278a74 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.061298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.061976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.062263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.062545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.074989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:01.081043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:01.081941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:01.083148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 46/92] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:01.090195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.176024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 47/93] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:01.177510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.182773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:01.204351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9466172e-e4ab-4f0d-b1ce-f92c57296c61 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.256436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.257048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.257343 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.257628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.260586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:01.260586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:01.260586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:01.278953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 47/94] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:01.285254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.367121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:01.371537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e29ebd-b661-4f3c-85cd-1de977e99985 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.416612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.418720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.419220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.419689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.422079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a5af3-1fee-40c4-ac49-04bc6b1e75ef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.422919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:01.425443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:01.428302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 48/95] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:01.433639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.521332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.521332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.521332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.521332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] PATH_INFO: `/v3/users/8eaffdb83dc6462bb58ec88e4b1c7de8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.543235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:01.549230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] RBAC: Authorizing `identity:get_user(user_id=8eaffdb83dc6462bb58ec88e4b1c7de8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:01.552093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:01.552093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 48/96] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:01 2022] GET /identity/v3/users/8eaffdb83dc6462bb58ec88e4b1c7de8 => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:01.558273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.629279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73885044-786d-44d3-9c25-aa18ac05ee15 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.692370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.693096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.693427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.693711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.694437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:01.695931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:01.737097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624afed-65e4-4f7c-96d4-7e71f4bdabf8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.755069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 49/97] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:18:01 2022] POST /identity/v3/projects => generated 302 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:18:01.755982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.783445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.784371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.784704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.784986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.786355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:01.788631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:01.790396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 49/98] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:01 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:01.796184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:01.806434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:01.877577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:01.925130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076446be-5e55-40b2-8060-23b6dc5dd10c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:01.967229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f36263-08d1-4156-9e4c-1e8c128d968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:01.968129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36263-08d1-4156-9e4c-1e8c128d968e None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.968561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36263-08d1-4156-9e4c-1e8c128d968e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:01.968994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f36263-08d1-4156-9e4c-1e8c128d968e None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5/users/8eaffdb83dc6462bb58ec88e4b1c7de8/roles/31deee9228fc4ecd930c156b89edb5c6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:01.972455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f36263-08d1-4156-9e4c-1e8c128d968e None admin] RBAC: Authorizing `identity:create_grant(project_id=43e403631cd6479489b75c17c38466d5, user_id=8eaffdb83dc6462bb58ec88e4b1c7de8, role_id=31deee9228fc4ecd930c156b89edb5c6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:01.974265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f36263-08d1-4156-9e4c-1e8c128d968e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:01.996122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 50/99] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:01 2022] PUT /identity/v3/projects/43e403631cd6479489b75c17c38466d5/users/8eaffdb83dc6462bb58ec88e4b1c7de8/roles/31deee9228fc4ecd930c156b89edb5c6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:01.998535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481212d-53f5-4f3c-90d6-86992d60a8fd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:01.999519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481212d-53f5-4f3c-90d6-86992d60a8fd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.000316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4481212d-53f5-4f3c-90d6-86992d60a8fd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.000699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b329ffc8-4de1-4171-a3ee-6306bb0133f3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:02.003282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 51/100] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:02.009819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.010367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.010786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.058551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:02.059796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:02.071842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f5344-3816-44d8-939e-66889794b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:02.072510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5344-3816-44d8-939e-66889794b75f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.072796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5344-3816-44d8-939e-66889794b75f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.073071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f5344-3816-44d8-939e-66889794b75f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.089821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f5344-3816-44d8-939e-66889794b75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-602f5344-3816-44d8-939e-66889794b75f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:02.094947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:02.096731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:02.097647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 50/101] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:02.098837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7841b8a-3db0-465b-b340-faa957881937 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.099128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7841b8a-3db0-465b-b340-faa957881937 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.099408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7841b8a-3db0-465b-b340-faa957881937 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.101007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 51/102] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:02.107871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13abc4-f21d-4ded-b7de-5a6b6daa0b1c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.108156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13abc4-f21d-4ded-b7de-5a6b6daa0b1c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.108437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13abc4-f21d-4ded-b7de-5a6b6daa0b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.142762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ba13abc4-f21d-4ded-b7de-5a6b6daa0b1c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:02.144292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ba13abc4-f21d-4ded-b7de-5a6b6daa0b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:02.158872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 52/103] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:02.159899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:02.235764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:02.273869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 52/104] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:02.275512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f298bb-0e89-4a47-82c1-7ed3d7e41c92 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.275943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f298bb-0e89-4a47-82c1-7ed3d7e41c92 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.276392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f298bb-0e89-4a47-82c1-7ed3d7e41c92 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.278032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 53/105] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:02.284107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.284585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.285022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.306780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40725d4-6a52-40eb-9e85-207c1ba66c16 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:02.308761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:02.310890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:02.319159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:02.353027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:02.353675 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.353955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.354231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.354895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:02.355197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:02.357112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:02.368926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 53/106] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:02.376497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:02.386701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 54/107] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:02.480994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f7c4b-9510-4a35-8f09-b164b1d697e2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:02.502992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:02.517440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:02.518593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.519149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.519737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] PATH_INFO: `/v3/users/c0edd4c51fd342338cf2652e2ee1df11` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.521230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] RBAC: Authorizing `identity:get_user(user_id=c0edd4c51fd342338cf2652e2ee1df11)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:02.522713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:02.524590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 54/108] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:02 2022] GET /identity/v3/users/c0edd4c51fd342338cf2652e2ee1df11 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:02.530115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:02.636384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38084406-b0a8-452b-b1df-8140361efe92 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:02.672025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:02.672727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.674333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:02.681680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.682274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.693226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:02.791329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:02.795833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:02.817639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1129f8dc-e456-40a1-a26e-8ee30348df8d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:02.862081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:02.862722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:02.863004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:02.863277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:02.874382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:02.880319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:02.881162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:02.881976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 55/109] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:02.886162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:02.935196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 55/110] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:02.936369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:02.946873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:02.977842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.011283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e71e7-5f99-4b75-9a61-1a6598a22caa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.046729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb62bf-d942-48c6-8380-ae10912a7088 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.057949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.058718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.059627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.059995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.076820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:03.084270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:03.085190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:03.086009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 56/111] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:18:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:03.086760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.091002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.091681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.092208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.092413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.110543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:03.116495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:03.117404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:03.118204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 56/112] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:03.118907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.215284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984fab5-9a7f-4383-b87d-4f4d79438550 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.224182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.249797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.254416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564bcc2-e9ff-4278-871c-6ea26173c6c2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.259933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.261439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.261439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.261724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.262467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:03.262842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.266724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.281595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 57/113] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.281595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.301666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.302230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.302512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.302791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.304063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.305566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.307389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 57/114] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.307939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.408706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ce4f3-b211-43df-a4b2-94d5e9246203 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.409211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c912d-e636-4609-9eca-2727d5acc39e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.443449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.444062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.444344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.444624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.445295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:03.445628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.447015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.450064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97314440-5432-4edc-bb48-c7542994030f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.450741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97314440-5432-4edc-bb48-c7542994030f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.451094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97314440-5432-4edc-bb48-c7542994030f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.451440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97314440-5432-4edc-bb48-c7542994030f None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.452749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97314440-5432-4edc-bb48-c7542994030f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:03.453138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97314440-5432-4edc-bb48-c7542994030f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.455496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97314440-5432-4edc-bb48-c7542994030f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.456249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 58/115] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.456865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.473460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 58/116] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.474359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97314440-5432-4edc-bb48-c7542994030f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.493076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.565770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.592547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0cd83-b375-4ba3-b5ec-41e10402a99f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.593701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97314440-5432-4edc-bb48-c7542994030f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.630950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.631590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.631873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.632151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] PATH_INFO: `/v3/users/c0edd4c51fd342338cf2652e2ee1df11` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.633435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] RBAC: Authorizing `identity:get_user(user_id=c0edd4c51fd342338cf2652e2ee1df11)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.635002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.636946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 59/117] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/users/c0edd4c51fd342338cf2652e2ee1df11 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.638902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.639639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.640018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.640389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.641213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] RBAC: Adding query filter params (role.id=0e327db8aba54600aac0165be40f349b, scope.project.id=43e403631cd6479489b75c17c38466d5, user.id=c0edd4c51fd342338cf2652e2ee1df11) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:03.641654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.642210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.643449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.651067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 59/118] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/role_assignments?user.id=c0edd4c51fd342338cf2652e2ee1df11&scope.project.id=43e403631cd6479489b75c17c38466d5&role.id=0e327db8aba54600aac0165be40f349b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.652324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.768448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.775574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf731b0-cf5c-4a67-8d43-b772c8d37461 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.819388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.820102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.820767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.821151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] PATH_INFO: `/v3/users/c0edd4c51fd342338cf2652e2ee1df11` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.822923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] RBAC: Authorizing `identity:get_user(user_id=c0edd4c51fd342338cf2652e2ee1df11)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.824504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.827127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 60/119] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/users/c0edd4c51fd342338cf2652e2ee1df11 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.827873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.858850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:03.887363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7df0e1-e98f-48ad-9888-98ffac6ef8b9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:03.931976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:03.932618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:03.932905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:03.933186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] PATH_INFO: `/v3/users/8feb61a946fd40a291adfe0eb3820253` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:03.934369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] RBAC: Authorizing `identity:get_user(user_id=8feb61a946fd40a291adfe0eb3820253)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:03.935841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:03.937520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 60/120] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/users/8feb61a946fd40a291adfe0eb3820253 => generated 247 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:03.938360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:03.955996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f09b-c096-485d-a818-78eb5b24bc55 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.005369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.005998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.006559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.006844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.047099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.065270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0cd17-b4a0-4207-892c-364ef210214d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.101508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.103248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.109556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.110182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.110983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.111279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.144447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.187416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.189491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.230636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 61/121] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:04.231388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.308122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.312430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 61/122] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:04.313976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.353532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b09a5d-70b3-44e4-a79b-df78a1d41bb7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.392502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.393673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.394645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.395233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.395796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.449007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40c52b8-54b2-45df-aae5-977c740ac020 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.479247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.480833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.498093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.498738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.499019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.499293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] PATH_INFO: `/v3/projects/6db4fff2382a4e01ab2e98ba9116f1be` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.500551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] RBAC: Authorizing `identity:get_project(project_id=6db4fff2382a4e01ab2e98ba9116f1be)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.502715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.504560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 62/123] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/projects/6db4fff2382a4e01ab2e98ba9116f1be => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:04.505138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.567482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.609848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 62/124] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:04.610872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.622366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a746e15-6c8d-4503-b135-6955bc99be77 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.652712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.658330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c1074-344e-4fba-ae1a-08eeb3b8d867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.658971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1074-344e-4fba-ae1a-08eeb3b8d867 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.659257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1074-344e-4fba-ae1a-08eeb3b8d867 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.659534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c1074-344e-4fba-ae1a-08eeb3b8d867 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.660669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c1074-344e-4fba-ae1a-08eeb3b8d867 None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.662158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c1074-344e-4fba-ae1a-08eeb3b8d867 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.663780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 63/125] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:04.666609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada408-dbda-4354-90df-9ed28bdec467 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.666901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada408-dbda-4354-90df-9ed28bdec467 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.667175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbada408-dbda-4354-90df-9ed28bdec467 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.668423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 64/126] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:04.669341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbada408-dbda-4354-90df-9ed28bdec467 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.722886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e69045-6526-4826-a7e9-c09990ed23cd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.756380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.757067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.757382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.757664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.758382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] RBAC: Adding query filter params (role.id=0e327db8aba54600aac0165be40f349b, scope.project.id=6db4fff2382a4e01ab2e98ba9116f1be, user.id=c0edd4c51fd342338cf2652e2ee1df11) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:04.758691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.760132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550c6aa-b38e-43ec-870a-06dfa7b84316 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.765657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 63/127] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/role_assignments?user.id=c0edd4c51fd342338cf2652e2ee1df11&scope.project.id=6db4fff2382a4e01ab2e98ba9116f1be&role.id=0e327db8aba54600aac0165be40f349b => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:04.766832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53b00-5134-42c1-95cf-b4d7cc911fe4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.767654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53b00-5134-42c1-95cf-b4d7cc911fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.767955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e53b00-5134-42c1-95cf-b4d7cc911fe4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.769244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 64/128] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:04.770009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e53b00-5134-42c1-95cf-b4d7cc911fe4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.772972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbada408-dbda-4354-90df-9ed28bdec467 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.805821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.823176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24029bf6-a89d-4302-b554-38a90297f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.824159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24029bf6-a89d-4302-b554-38a90297f358 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.824623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24029bf6-a89d-4302-b554-38a90297f358 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.825543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24029bf6-a89d-4302-b554-38a90297f358 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.827570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24029bf6-a89d-4302-b554-38a90297f358 None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.829781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24029bf6-a89d-4302-b554-38a90297f358 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.832550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 65/129] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:04.834479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49a8d-7cd8-4ca7-99b7-0bec059deed8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.834903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49a8d-7cd8-4ca7-99b7-0bec059deed8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.835309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e49a8d-7cd8-4ca7-99b7-0bec059deed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.856624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-41e49a8d-7cd8-4ca7-99b7-0bec059deed8 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:04.858017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-41e49a8d-7cd8-4ca7-99b7-0bec059deed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:04.893491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e53b00-5134-42c1-95cf-b4d7cc911fe4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:04.931802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:04.939257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 66/130] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:04.940709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.940999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.941335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.951937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:04.951937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:04.951937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:04.951937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:04.952272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] RBAC: Adding query filter params (role.id=2b9a144dd3114f13a8fa1006f585fb12, scope.project.id=43e403631cd6479489b75c17c38466d5, user.id=c0edd4c51fd342338cf2652e2ee1df11) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:04.952272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:04.953256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:04.960111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 65/131] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/role_assignments?user.id=c0edd4c51fd342338cf2652e2ee1df11&scope.project.id=43e403631cd6479489b75c17c38466d5&role.id=2b9a144dd3114f13a8fa1006f585fb12 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:04.960928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:04.964486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:04.966062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:05.058969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 67/132] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:05.093511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d895-bf29-4cf9-bd72-77fd4d7b9986 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.095927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:05.133587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.134224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.134509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.134785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:05.135533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=43e403631cd6479489b75c17c38466d5, user.id=8feb61a946fd40a291adfe0eb3820253) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:05.135844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:05.137346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:05.144052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 66/133] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:04 2022] GET /identity/v3/role_assignments?user.id=8feb61a946fd40a291adfe0eb3820253&scope.project.id=43e403631cd6479489b75c17c38466d5&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:05.274290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:05.349559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:05.383640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca10652-3d10-483e-9731-0fb9233ec48d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.413964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:05.421882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.422512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.422800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.423621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:05.424922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:05.426416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:05.429335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 68/134] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:05.433382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:05.458378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed84827-3ce1-4789-8070-ddc0ac05dc0d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.491923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.492613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.492824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.493097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:05.503245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:05.509151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:05.510050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:05.510833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 67/135] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:05.515898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:05.533962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:05.537387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb2d6e6-2b60-48c2-95f8-43d23df496dd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.570692 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.571289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.571572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.571847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:05.572494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:05.573981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:05.600410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 69/136] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:05 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:18:05.623509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc62d9-22b3-4daf-aad0-b70286259fc8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.659982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.660612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.660890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.661158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:05.661874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:05.662180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:05.663611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:05.672687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 68/137] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:05.679259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:05.734934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:05.785326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f68712-23aa-427c-ad3c-d1b11051fd16 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.820185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.820825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.821102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.821444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] PATH_INFO: `/v3/users/8eaffdb83dc6462bb58ec88e4b1c7de8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:05.822568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] RBAC: Authorizing `identity:get_user(user_id=8eaffdb83dc6462bb58ec88e4b1c7de8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:05.824014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:05.825608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 70/138] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:05 2022] GET /identity/v3/users/8eaffdb83dc6462bb58ec88e4b1c7de8 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:05.831227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:05.915752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:05.940891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df543ef-3bbf-4e42-a91f-e84a97a0568a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:05.977256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593601-3886-452c-9401-a094de086c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:05.977901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593601-3886-452c-9401-a094de086c4a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:05.978186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593601-3886-452c-9401-a094de086c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:05.978460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af593601-3886-452c-9401-a094de086c4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:06.052391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af593601-3886-452c-9401-a094de086c4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:06.054134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af593601-3886-452c-9401-a094de086c4a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:06.136113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:06.162868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 69/139] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:06.168691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:06.284077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2c2bd5-d98c-4c98-8a9c-1f93b0205b9f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:06.308977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:06.324797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e0dbe-82ab-4972-b1f3-71b7f3593174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:06.325770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0dbe-82ab-4972-b1f3-71b7f3593174 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:06.326252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0dbe-82ab-4972-b1f3-71b7f3593174 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:06.326710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e0dbe-82ab-4972-b1f3-71b7f3593174 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:06.328032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e0dbe-82ab-4972-b1f3-71b7f3593174 None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:06.329847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e0dbe-82ab-4972-b1f3-71b7f3593174 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:06.331688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 71/140] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:06 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:06.338459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593601-3886-452c-9401-a094de086c4a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:06.445641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af593601-3886-452c-9401-a094de086c4a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:06.482801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:06.483733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:06.484198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:06.484663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:06.485630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] RBAC: Adding query filter params (role.id=31deee9228fc4ecd930c156b89edb5c6, scope.project.id=43e403631cd6479489b75c17c38466d5, user.id=8eaffdb83dc6462bb58ec88e4b1c7de8) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:06.486301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:06.487915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a364e919-84c6-45ea-8964-c57e1ca29c27 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:06.499243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 70/141] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:06 2022] GET /identity/v3/role_assignments?user.id=8eaffdb83dc6462bb58ec88e4b1c7de8&scope.project.id=43e403631cd6479489b75c17c38466d5&role.id=31deee9228fc4ecd930c156b89edb5c6 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:07.848751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dfa8d-0488-4e7c-8867-8fd0b47ac3f7 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:07.849284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dfa8d-0488-4e7c-8867-8fd0b47ac3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:07.849284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2dfa8d-0488-4e7c-8867-8fd0b47ac3f7 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:07.850824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 72/142] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:07.855751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9261b-07fe-4c6a-86e5-77df5031a7bd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:07.856148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9261b-07fe-4c6a-86e5-77df5031a7bd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:07.856310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb9261b-07fe-4c6a-86e5-77df5031a7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:07.875084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3bb9261b-07fe-4c6a-86e5-77df5031a7bd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:07.876871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3bb9261b-07fe-4c6a-86e5-77df5031a7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:07.912107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:07.951507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 71/143] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:08.037511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e452e-8adb-459c-b755-a86d3139255a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.037792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e452e-8adb-459c-b755-a86d3139255a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.038263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e452e-8adb-459c-b755-a86d3139255a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.039651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 73/144] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:08.045742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e631808-5aa2-44b2-9f1b-ebeb806048ef None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.046049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e631808-5aa2-44b2-9f1b-ebeb806048ef None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.046292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e631808-5aa2-44b2-9f1b-ebeb806048ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.068078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6e631808-5aa2-44b2-9f1b-ebeb806048ef None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:08.069264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6e631808-5aa2-44b2-9f1b-ebeb806048ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:08.139394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 72/145] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:08.142991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af728218-4eb9-4fa1-a014-9ca4edf3c9ae None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.143282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af728218-4eb9-4fa1-a014-9ca4edf3c9ae None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.143565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af728218-4eb9-4fa1-a014-9ca4edf3c9ae None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.145013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 74/146] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:08.150008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.150251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.150527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.168773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:08.169984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:08.217545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af728218-4eb9-4fa1-a014-9ca4edf3c9ae None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.241085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 73/147] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:08.326308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af728218-4eb9-4fa1-a014-9ca4edf3c9ae None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.361431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.362048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.362327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.362597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.372995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:08.377859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:08.378734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:08.380082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 75/148] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:08.384759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.417244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.420400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:08.473435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:08.495753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7587df-fc67-4328-b001-fef0a2612918 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.525232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05339b9b-bfba-48bf-b57a-14268fed8054 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.531502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.532139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.533064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.533369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.534035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:08.534382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:08.535757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:08.545544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 74/149] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:08.546332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.563755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.564382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.564660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.564930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.574799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:08.580167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:08.581043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 76/150] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:08.586254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.636831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:08.646934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5404b35b-4faf-4f46-97c6-71f7d0060474 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.680196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.680544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23b22ce-5f6f-4ccb-b6dc-414751eeb1e8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.680756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.681039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.681330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.682443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:08.683949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:08.685555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 75/151] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:08.686323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.691556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:08.718373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.719000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.719281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.719560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] PATH_INFO: `/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.733105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] RBAC: Authorizing `identity:get_user(user_id=ae3546e9d1f3469c9c246f6e5e498a2d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:08.734611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:08.736619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 77/152] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:08.737176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.787693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb957fd-4376-45c5-b8c0-6a788d34276a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.820308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.820911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.821187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.821487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.822145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:08.822446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:08.823808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:08.832205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 76/153] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:08.833084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1306375f-e98a-4b4b-8ecf-8d3b9b13c593 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.834193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.860846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:08.866875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:08.867492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:08.867776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:08.868053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:08.868739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:08.870211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:08.890605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 78/154] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:08 2022] POST /identity/v3/users => generated 297 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:08.891419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:08.920452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:08.958720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560d8591-826d-4f4d-a2a2-ec64ce56e0ff None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:08.999713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:09.000369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:09.000638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:09.000943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:09.002229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:09.003712 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:09.005575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 77/155] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:09.009590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45d7f2-d9cb-42b6-8cae-7ea6b0cfa97c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:09.011266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:09.051069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:09.051721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:09.052006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:09.052283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] PATH_INFO: `/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:09.053510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] RBAC: Authorizing `identity:get_user(user_id=ae3546e9d1f3469c9c246f6e5e498a2d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:09.055054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:09.056840 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 79/156] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:08 2022] GET /identity/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:09.062343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:09.118210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:09.131830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048c3ca9-01eb-42a1-92a2-d5ff3122e122 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:09.168977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:09.182154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:09.182481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67576026-b72a-4cd8-9a5b-db137b923e39 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:09.182819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:09.183113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:09.183357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/ae3546e9d1f3469c9c246f6e5e498a2d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:09.186026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] RBAC: Authorizing `identity:create_grant(project_id=731ddd9bfa3c4ac7947beb04adaef28c, user_id=ae3546e9d1f3469c9c246f6e5e498a2d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:09.187473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:09.204127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 78/157] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:09 2022] PUT /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/ae3546e9d1f3469c9c246f6e5e498a2d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:18:09.228104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82badb9-0ab1-4203-8fba-abc961ee203e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:09.228771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82badb9-0ab1-4203-8fba-abc961ee203e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:09.229054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82badb9-0ab1-4203-8fba-abc961ee203e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:09.229377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82badb9-0ab1-4203-8fba-abc961ee203e None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:09.230898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82badb9-0ab1-4203-8fba-abc961ee203e None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:09.232431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82badb9-0ab1-4203-8fba-abc961ee203e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:09.234240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 80/158] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:09 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:09.241748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:09.365632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775c73-f1b1-4f44-8a5f-3bb57da8ab2b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:09.403568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f366d20-a48c-4232-bbe9-0de7735178e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:09.404677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366d20-a48c-4232-bbe9-0de7735178e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:09.405159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366d20-a48c-4232-bbe9-0de7735178e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:09.405670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f366d20-a48c-4232-bbe9-0de7735178e8 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/ae3546e9d1f3469c9c246f6e5e498a2d/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:09.408160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366d20-a48c-4232-bbe9-0de7735178e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=731ddd9bfa3c4ac7947beb04adaef28c, user_id=ae3546e9d1f3469c9c246f6e5e498a2d, role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:09.409868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f366d20-a48c-4232-bbe9-0de7735178e8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:09.426148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 79/159] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:09 2022] PUT /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/ae3546e9d1f3469c9c246f6e5e498a2d/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:11.181199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bcff7-9459-4419-a3ef-2d505096feb2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.181738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bcff7-9459-4419-a3ef-2d505096feb2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.181813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500bcff7-9459-4419-a3ef-2d505096feb2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.183254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 81/160] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:11.188591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecdbb8-05d6-4087-a2d5-3f588ee6371c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.189028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecdbb8-05d6-4087-a2d5-3f588ee6371c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.189223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecdbb8-05d6-4087-a2d5-3f588ee6371c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.208380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-94ecdbb8-05d6-4087-a2d5-3f588ee6371c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:11.209923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-94ecdbb8-05d6-4087-a2d5-3f588ee6371c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:11.213089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:11.292159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 80/161] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:11.347176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9482b1-89bf-4df7-91e3-f0bb53bf4f4c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.347520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9482b1-89bf-4df7-91e3-f0bb53bf4f4c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.347736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9482b1-89bf-4df7-91e3-f0bb53bf4f4c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.349130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 82/162] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:11.355503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa43f20-0732-4d38-813a-0f14f642cbfb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.355804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa43f20-0732-4d38-813a-0f14f642cbfb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.356060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa43f20-0732-4d38-813a-0f14f642cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.375455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bfa43f20-0732-4d38-813a-0f14f642cbfb None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:11.376752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bfa43f20-0732-4d38-813a-0f14f642cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:11.449483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 81/163] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:18:11.451077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138c359-3808-4d00-8ed5-47ebf9a5589e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.451355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138c359-3808-4d00-8ed5-47ebf9a5589e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.451623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138c359-3808-4d00-8ed5-47ebf9a5589e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.452950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 83/164] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:11.458658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9befab-a4d8-46d5-93ee-4689eaa5895b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.458943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9befab-a4d8-46d5-93ee-4689eaa5895b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.459211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9befab-a4d8-46d5-93ee-4689eaa5895b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.477239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8f9befab-a4d8-46d5-93ee-4689eaa5895b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:11.478442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8f9befab-a4d8-46d5-93ee-4689eaa5895b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:11.541767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:11.546654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 82/165] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:11.562696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138c359-3808-4d00-8ed5-47ebf9a5589e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:11.675207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138c359-3808-4d00-8ed5-47ebf9a5589e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:11.702619 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3792-e161-4aad-90ab-3aabad85122d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.703599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3792-e161-4aad-90ab-3aabad85122d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.704939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3792-e161-4aad-90ab-3aabad85122d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.706642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 83/166] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:18:11.711734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.711880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:11.712380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.712608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.712705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.712909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.713403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.725882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:11.731489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:11.732363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:11.733160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 84/167] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:11.733901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:11.735794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:11.737038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:11.758653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:11.809349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 84/168] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:18:11.809946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:11.849533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1842ec-1c49-44ad-85d1-6b33931f326f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:11.886406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:11.887027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.887301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.887568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.888662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:11.890642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:11.893897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 85/169] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:11.894787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:11.915470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d8ff3-56bc-44d9-8d92-a1d1ca66aae9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:11.948323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:11.955664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:11.956203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:11.956482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:11.956751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:11.957443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:11.957753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:11.959160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:11.971640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 85/170] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:11.971640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:11.995825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.012054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d82d5-e685-4bbf-8829-4f4bb3e0390b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.045470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.046141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.046417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.046687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.057133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:12.062593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:12.064055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:12.064835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 86/171] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:12.069707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.080438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab86cd7d-35c6-4da9-8d5e-5b42daf05ed9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.118330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.119223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.119508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.119783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.120432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.122607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.143581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 86/172] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:11 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:12.144383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.176435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b62892-7e16-4df7-b3a4-b8495eda3253 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.202793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.215735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.216284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.216671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.216858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] PATH_INFO: `/v3/users/8e5993932cb9454b94af0a003a495e4c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.232804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] RBAC: Authorizing `identity:get_user(user_id=8e5993932cb9454b94af0a003a495e4c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.234348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.236735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 87/173] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/users/8e5993932cb9454b94af0a003a495e4c => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:12.236980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.242055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.283263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721233cd-e710-4ee9-9eb5-977db7e09397 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.324666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.325547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.325820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.326088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.326761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:12.327075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.328432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.338168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 87/174] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:12.339755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.357488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ea9e-f4c6-4c62-a6f8-d9bad32ad515 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.399472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.400115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.400398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.400673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.413297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:12.419491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:12.420396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:12.421185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 88/175] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:12.426351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.460431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597686de-eef0-4b36-8eac-ab7d652c0db7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.473339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.506621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.507270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.507546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.507819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.508236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.509426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.511132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.513723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 88/176] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:12.514315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.557958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b90c2bc-8494-4978-93f7-4b232716bf90 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.598084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.599206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.600204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.600687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] PATH_INFO: `/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.602374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] RBAC: Authorizing `identity:get_user(user_id=ae3546e9d1f3469c9c246f6e5e498a2d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.605094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.607037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 89/177] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:12.607992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.641519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f8888-96e3-436c-b0ad-62a1b26c747e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.690457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.691079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.691359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.691633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.692567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:12.692885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.695116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.706423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 89/178] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:12.710117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.743550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce7383f-6b3a-485d-8466-d73f680cd70a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.774508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.781803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.783078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.783362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.783638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/8e5993932cb9454b94af0a003a495e4c/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.785696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] RBAC: Authorizing `identity:create_grant(project_id=731ddd9bfa3c4ac7947beb04adaef28c, user_id=8e5993932cb9454b94af0a003a495e4c, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:12.787213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:12.801885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 90/179] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:12 2022] PUT /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/8e5993932cb9454b94af0a003a495e4c/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:12.802895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:12.803031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:12.826960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f518b-6cb3-4443-93bb-c6d7666022cb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:12.865600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:12.866620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:12.867113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:12.867600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:12.998518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:13.000383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:13.019671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6b13e-0621-40b5-ab08-fb9e22b19406 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:13.044243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:13.064128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:13.065796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:13.066410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:13.066975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] PATH_INFO: `/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:13.069008 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] RBAC: Authorizing `identity:get_user(user_id=ae3546e9d1f3469c9c246f6e5e498a2d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:13.071009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:13.073227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 91/180] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/users/ae3546e9d1f3469c9c246f6e5e498a2d => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:13.078607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:13.120965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 90/181] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:13.127971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:13.153762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:13.238238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603edd99-b2bb-4a47-85c1-3760c5631bed None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:13.291529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:13.292648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:13.293262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:13.293778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:13.316546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d70b1ec-8107-454e-8783-0a74258fc3f5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:13.366309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:13.366959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:13.367235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:13.367505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:13.368671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:13.370203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:13.375029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 91/182] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:13 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:13.379175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:13.400336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:13.425934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:13.427430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:13.515020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:13.548398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf6f4ff-b61c-4a08-b8b6-a05f90e5cb9a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:13.584150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:13.585065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:13.585371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:13.585654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:13.586442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c, user.id=ae3546e9d1f3469c9c246f6e5e498a2d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:13.586758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:13.588218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:13.595322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 92/183] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:13 2022] GET /identity/v3/role_assignments?user.id=ae3546e9d1f3469c9c246f6e5e498a2d&scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:13.622781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 92/184] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:13.629100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:13.735410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:13.773573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e57ea4-7dbc-43e9-99aa-92c52239b5df None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:13.821498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bd8b4a-54c1-440d-83a7-5c55741beddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:13.822553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd8b4a-54c1-440d-83a7-5c55741beddc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:13.823062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd8b4a-54c1-440d-83a7-5c55741beddc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:13.823540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bd8b4a-54c1-440d-83a7-5c55741beddc None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:13.824906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd8b4a-54c1-440d-83a7-5c55741beddc None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:13.826583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bd8b4a-54c1-440d-83a7-5c55741beddc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:13.828432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 93/185] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:13 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:13.835498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:13.956684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b686b0a6-bdc0-4f54-af6a-958ae5452b9a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:13.996061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:13.996679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:13.997218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:13.997537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:13.998255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] RBAC: Adding query filter params (role.id=2b9a144dd3114f13a8fa1006f585fb12, scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c, user.id=ae3546e9d1f3469c9c246f6e5e498a2d) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:13.998564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:14.000025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29026b-e946-4358-b264-bbcb812f6344 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:14.007744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 93/186] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:13 2022] GET /identity/v3/role_assignments?user.id=ae3546e9d1f3469c9c246f6e5e498a2d&scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c&role.id=2b9a144dd3114f13a8fa1006f585fb12 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:14.571113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e90837-c70f-43bd-a81a-3b177d9db861 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:14.571599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e90837-c70f-43bd-a81a-3b177d9db861 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:14.571599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e90837-c70f-43bd-a81a-3b177d9db861 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:14.573051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 94/187] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:14.578002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f22b84-aabd-4e2d-a6da-f4311a240e8b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:14.578394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f22b84-aabd-4e2d-a6da-f4311a240e8b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:14.578550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f22b84-aabd-4e2d-a6da-f4311a240e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:14.597851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-48f22b84-aabd-4e2d-a6da-f4311a240e8b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:14.599049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-48f22b84-aabd-4e2d-a6da-f4311a240e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:14.617591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:14.668060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 94/188] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:14.866121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f603d-746c-4be8-85ab-4e04e3350d70 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:14.866451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f603d-746c-4be8-85ab-4e04e3350d70 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:14.866645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787f603d-746c-4be8-85ab-4e04e3350d70 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:14.868056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 95/189] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:14.873401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:14.873688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:14.873959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:14.892450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:14.893631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:14.935753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f603d-746c-4be8-85ab-4e04e3350d70 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:14.963397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 95/190] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:15.039936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787f603d-746c-4be8-85ab-4e04e3350d70 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.073084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.073735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.074014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.074284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.082285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:15.086298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:15.091568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:15.092682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:15.093721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:15.097741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 96/191] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:15.101798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.192696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:15.210289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473abd8f-57ba-4411-b50a-159a373a5a63 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.228440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.251700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.252327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.252608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.252880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.254061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:15.254372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:15.255766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd9676c-e873-473e-9122-3399d1fbcd33 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:15.273296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 96/192] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:15.273296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694279f-1b02-4501-a2aa-db462d13f166 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.273296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694279f-1b02-4501-a2aa-db462d13f166 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.273296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694279f-1b02-4501-a2aa-db462d13f166 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.273296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 97/193] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:15.274765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694279f-1b02-4501-a2aa-db462d13f166 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.334478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08d99c4-e0f1-498c-803b-cc3bf6c1928f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.367969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f0eda-fe7e-4d6e-aee3-9803e095c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.368596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0eda-fe7e-4d6e-aee3-9803e095c898 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.368874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0eda-fe7e-4d6e-aee3-9803e095c898 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.369146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f0eda-fe7e-4d6e-aee3-9803e095c898 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.370285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f0eda-fe7e-4d6e-aee3-9803e095c898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:15.371988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f0eda-fe7e-4d6e-aee3-9803e095c898 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:15.373531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694279f-1b02-4501-a2aa-db462d13f166 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.373636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 97/194] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:15.374684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.374964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.375235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.393179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:15.394411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:15.411824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.412464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.412741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.413013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] PATH_INFO: `/v3/users/e40c79c9dc3e41398bb8859c557f9645` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.426513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] RBAC: Authorizing `identity:get_user(user_id=e40c79c9dc3e41398bb8859c557f9645)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:15.427968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:15.429606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 98/195] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:15 2022] GET /identity/v3/users/e40c79c9dc3e41398bb8859c557f9645 => generated 297 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:15.429606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.434289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:15.456898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:15.462917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 98/196] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:15.469194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.534364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38e0e81-ca78-4451-bbb4-2e9785258c42 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.569245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.569899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.570185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.570464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.571119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:15.572571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:15.575391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff0116-8ec5-487f-91aa-50afd74b21ce None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.593599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 99/197] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:15 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:15.622996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.623651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.623932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.624207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.625441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:15.627511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:15.629447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 99/198] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:15 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:15.635360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.705570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:15.734001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:15.758119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777d8cf2-7e47-432f-93ec-0db3d76b2f22 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.798922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.799586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.799869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.800144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/e40c79c9dc3e41398bb8859c557f9645/roles/31deee9228fc4ecd930c156b89edb5c6` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.802533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] RBAC: Authorizing `identity:create_grant(project_id=731ddd9bfa3c4ac7947beb04adaef28c, user_id=e40c79c9dc3e41398bb8859c557f9645, role_id=31deee9228fc4ecd930c156b89edb5c6)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:15.804157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:15.813634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:15.819280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 100/199] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:15 2022] PUT /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/users/e40c79c9dc3e41398bb8859c557f9645/roles/31deee9228fc4ecd930c156b89edb5c6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:15.865406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627383-1374-432a-9f01-b64ddae70554 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:15.909661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:15.910313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:15.910589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:15.910858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:15.923365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:15.928813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:15.929962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:15.930949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:15.933132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 100/200] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:15.941450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:16.053750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a01b7-6961-479e-a42d-c731cf21d64a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:16.076162 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:16.096144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:16.096757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:16.097037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:16.097329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:16.098008 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:16.098309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:16.099688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:16.110508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 101/201] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:16.118411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:16.231618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d65b14-0531-4cbc-a5dd-1549cbef2f7b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:16.260514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:16.273999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:16.274632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:16.274905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:16.275171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] PATH_INFO: `/v3/users/8e5993932cb9454b94af0a003a495e4c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:16.276306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=8e5993932cb9454b94af0a003a495e4c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:16.277811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:16.279552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 101/202] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:16 2022] GET /identity/v3/users/8e5993932cb9454b94af0a003a495e4c => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:16.285461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:16.405022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e0bfa-2a46-4963-acfb-3a734c65c63c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:16.441204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:16.441846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:16.442126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:16.442402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:16.494934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:16.515310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:16.517410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:16.621592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 102/203] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:16.630376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:16.744096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa30ed-61c2-4cc8-a823-c1af75aca1d2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:16.781998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6b393-5f8b-45f5-a989-2ffac5152354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:16.782905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6b393-5f8b-45f5-a989-2ffac5152354 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:16.783373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6b393-5f8b-45f5-a989-2ffac5152354 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:16.783828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6b393-5f8b-45f5-a989-2ffac5152354 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:16.785143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6b393-5f8b-45f5-a989-2ffac5152354 None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:16.786796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a6b393-5f8b-45f5-a989-2ffac5152354 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:16.793479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 102/204] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:16 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:16.795224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:16.882624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:16.904475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5ed27-4023-4815-b5eb-c64eaa22875e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:16.940860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:16.941501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:16.941784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:16.942057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:16.942786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] RBAC: Adding query filter params (role.id=0e327db8aba54600aac0165be40f349b, scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c, user.id=8e5993932cb9454b94af0a003a495e4c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:16.943262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:16.944722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85127bb8-124b-4589-b0f2-0945b072d0af None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:16.951180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 103/205] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:16 2022] GET /identity/v3/role_assignments?user.id=8e5993932cb9454b94af0a003a495e4c&scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c&role.id=0e327db8aba54600aac0165be40f349b => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:17.980055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50382-687a-4f57-ba09-6842029f9b1e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:17.980565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50382-687a-4f57-ba09-6842029f9b1e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:17.980640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d50382-687a-4f57-ba09-6842029f9b1e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:17.982093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 103/206] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:17.987422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed245cb-b8ed-43f6-a56a-a4bbc1be7c1e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:17.987834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed245cb-b8ed-43f6-a56a-a4bbc1be7c1e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:17.988016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed245cb-b8ed-43f6-a56a-a4bbc1be7c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.005608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2c421-4138-4fb4-8317-fb17d60b1a48 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.005937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2c421-4138-4fb4-8317-fb17d60b1a48 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.006163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b2c421-4138-4fb4-8317-fb17d60b1a48 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.007711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 104/207] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:18:18.007933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bed245cb-b8ed-43f6-a56a-a4bbc1be7c1e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:18.009187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bed245cb-b8ed-43f6-a56a-a4bbc1be7c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:18.013947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418924da-b69c-49a4-bdcb-25b7614e6aa9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.014251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418924da-b69c-49a4-bdcb-25b7614e6aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.014496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418924da-b69c-49a4-bdcb-25b7614e6aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.036919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-418924da-b69c-49a4-bdcb-25b7614e6aa9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:18.038168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-418924da-b69c-49a4-bdcb-25b7614e6aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:18.084041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 104/208] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:18.098236 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:18.109791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 105/209] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:18.123292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c6ff9-c7c5-4e9c-819f-d33d17147dd4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.123571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c6ff9-c7c5-4e9c-819f-d33d17147dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.123840 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c6ff9-c7c5-4e9c-819f-d33d17147dd4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.125330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 105/210] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:18.130704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.130995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.131267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.149831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:18.151002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:18.217339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 106/211] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:18.354600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c6ff9-c7c5-4e9c-819f-d33d17147dd4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.386441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.440611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:18.465992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c6ff9-c7c5-4e9c-819f-d33d17147dd4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.492443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2d2461-9d33-4f72-89c6-30212370c9de None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.502908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:18.503533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.503822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.504098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.514096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:18.515057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:18.519363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:18.520301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:18.521083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 106/212] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:18.521861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.529905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:18.530529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.530800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.531068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.532266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:18.533745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:18.535297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 107/213] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:18.536041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.622948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ecc912-5ced-42ac-9808-adc01e353d67 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.636123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fba6-df2f-4ee0-b72c-8625d23327ea None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.655833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:18.656456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.657381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.657682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.660501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:18.669274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:18.669880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:18.670482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.670761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.671034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.671689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:18.671990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:18.673936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:18.674677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:18.675710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:18.676718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 107/214] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:18.677546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.686625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 108/215] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:18.687894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.741175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:18.775021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1975a-2ef7-4a40-91ef-76f9faaadc74 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.788641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d49c5-c44b-4c4e-99e5-2adf3384aaee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.810683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:18.811280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.811584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.811852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.812495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:18.814465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:18.821868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:18.822469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:18.822745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:18.823022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:18.823712 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:18.824012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:18.825456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:18.833854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 109/216] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:18.834652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.837503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 108/217] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:18 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:18.846597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:18.914880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:18.959172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a29f3e-67f8-4d59-83b2-c606bb5fad82 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.971806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34f15e-8d75-4c62-9dba-e801f06afeab None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:18.974982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:19.004494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:19.005172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:19.005501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:19.005758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] PATH_INFO: `/v3/users/2d2ff3aaf6c246dc98bbd4cd488bfd64` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:19.013944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:19.015443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:19.015827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:19.016775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] PATH_INFO: `/v3/users/e40c79c9dc3e41398bb8859c557f9645` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:19.018261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] RBAC: Authorizing `identity:get_user(user_id=e40c79c9dc3e41398bb8859c557f9645)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:19.019969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:19.022354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 109/218] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/users/e40c79c9dc3e41398bb8859c557f9645 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:19.024044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] RBAC: Authorizing `identity:get_user(user_id=2d2ff3aaf6c246dc98bbd4cd488bfd64)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:19.025723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:19.027791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:19.033427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 110/219] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:18 2022] GET /identity/v3/users/2d2ff3aaf6c246dc98bbd4cd488bfd64 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:19.034714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:19.153670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c2663-7752-4b7c-b9d7-fd137e9fedd8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:19.179714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413a834-a0ad-419c-934b-21309f4f2b2e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:19.181902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:19.200237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:19.200901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:19.201177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:19.201575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:19.226302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:19.226506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:19.226784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:19.227050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:19.227773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] RBAC: Adding query filter params (role.id=0e327db8aba54600aac0165be40f349b, user.id=2d2ff3aaf6c246dc98bbd4cd488bfd64, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:19.228112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:19.230947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:19.237380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 111/220] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Oct 12 12:18:19 2022] GET /identity/v3/role_assignments?user.id=2d2ff3aaf6c246dc98bbd4cd488bfd64&scope.system=all&role.id=0e327db8aba54600aac0165be40f349b => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:19.296344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:19.298404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:19.441887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 110/221] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 12:18:19.448518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:19.501111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:19.581636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42715fe4-1092-4219-bfa2-0e7feb8fbdb3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:19.625767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf1c1f2-75e8-4541-b0ac-f79bc1331982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:19.626603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1c1f2-75e8-4541-b0ac-f79bc1331982 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:19.626879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1c1f2-75e8-4541-b0ac-f79bc1331982 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:19.627146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf1c1f2-75e8-4541-b0ac-f79bc1331982 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:19.628297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1c1f2-75e8-4541-b0ac-f79bc1331982 None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:19.629864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf1c1f2-75e8-4541-b0ac-f79bc1331982 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:19.631855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 112/222] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:19 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:19.638161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:19.670613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:19.759198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465c73fb-5e1b-4e65-b242-8e5deecb1124 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:19.801400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:19.802022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:19.802307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:19.802581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:19.803321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] RBAC: Adding query filter params (role.id=31deee9228fc4ecd930c156b89edb5c6, scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c, user.id=e40c79c9dc3e41398bb8859c557f9645) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:19.803628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:19.805078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec50c9c-bdab-4bbb-a1b4-9f71439bdc14 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:19.812130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 111/223] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:19 2022] GET /identity/v3/role_assignments?user.id=e40c79c9dc3e41398bb8859c557f9645&scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c&role.id=31deee9228fc4ecd930c156b89edb5c6 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:21.270049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681faee8-e8cc-43e0-afd1-9669a9d62546 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.270496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681faee8-e8cc-43e0-afd1-9669a9d62546 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.270588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681faee8-e8cc-43e0-afd1-9669a9d62546 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.271939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 113/224] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:21.277340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1603e9-b963-42f9-833f-ccb069dc2b66 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.278095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1603e9-b963-42f9-833f-ccb069dc2b66 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.278518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1603e9-b963-42f9-833f-ccb069dc2b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.297438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7f1603e9-b963-42f9-833f-ccb069dc2b66 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:21.298828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7f1603e9-b963-42f9-833f-ccb069dc2b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:21.373496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 112/225] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:21.455469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2463b-4a99-4d46-835e-b8055be393cb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.455780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2463b-4a99-4d46-835e-b8055be393cb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.456016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e2463b-4a99-4d46-835e-b8055be393cb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.457396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 114/226] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:21.462644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.462939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.463220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.482413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:21.483609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:21.493412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:21.533921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8cf08-b6dc-4edc-af20-a6644848dd3f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.534101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8cf08-b6dc-4edc-af20-a6644848dd3f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.534368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8cf08-b6dc-4edc-af20-a6644848dd3f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.535705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 115/227] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:21.540721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.541007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.541280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.552466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 113/228] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:21.559421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:21.560617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:21.628617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 116/229] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:18:21.658119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:21.755429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38dcf9-cd11-4fa8-8c9a-d01eae927b8f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:21.788440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:21.788993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.789267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.789562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.800170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:21.805055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:21.806097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:21.806895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 114/230] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:21.813396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:21.822544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:21.827334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:21.849829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:21.916085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f225d63-b755-4989-b3da-61b818c1a112 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:21.933569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63b7a0-2ac7-4a95-a079-aa5a319df30c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:21.954657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:21.955265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.955544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.955818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.956485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:21.956795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:21.958212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:21.966300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 117/231] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:21.967090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:21.972437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:21.973064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:21.973425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:21.973699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:21.974808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:21.976254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:21.977956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 115/232] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:21.978592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:22.046965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:22.064020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:22.070124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09df8e8a-7a1f-48af-bd7d-3b2b973599ec None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.077355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286f59d-bc54-41f4-982e-0dca12e4382a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.103997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.104607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.104880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.105154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.111592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.112201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.112486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.112761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] PATH_INFO: `/v3/users/35bb6e05f3074030bc1ca510baffdebb` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.115136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:22.120977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:22.122378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 118/233] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:22.123192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:22.128928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] RBAC: Authorizing `identity:get_user(user_id=35bb6e05f3074030bc1ca510baffdebb)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:22.130448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:22.132089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 116/234] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:21 2022] GET /identity/v3/users/35bb6e05f3074030bc1ca510baffdebb => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:22.137461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:22.223566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5ff458-5a21-4829-881c-38e702719269 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.235144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db597c33-5e4f-4ffa-9f40-8e740d794a5a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.255935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.256550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.256832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.257108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] PATH_INFO: `/v3/groups` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.257777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:22.259250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:22.267034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:22.274239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.274849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.275146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.275406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.276356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:22.276663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:22.277034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 119/235] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Oct 12 12:18:22 2022] POST /identity/v3/groups => generated 229 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:22.277667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:22.278100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:22.288544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 117/236] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:22.294664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:22.306506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:22.404977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef8f82-3f3e-496d-9a6f-97f617118274 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.407948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7785f-a05b-4046-87c6-62f73ea40b32 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.443111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.443726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.444012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.444286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.444781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.445018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] RBAC: Adding query filter params (role.id=31deee9228fc4ecd930c156b89edb5c6, user.id=35bb6e05f3074030bc1ca510baffdebb, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:22.445341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:22.445435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.445706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.445975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] PATH_INFO: `/v3/users/2d2ff3aaf6c246dc98bbd4cd488bfd64` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.446803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562703fe-f51d-4db6-b626-0d6a0681646f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:22.447093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] RBAC: Authorizing `identity:get_user(user_id=2d2ff3aaf6c246dc98bbd4cd488bfd64)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:22.449193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:22.450802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 118/237] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:22 2022] GET /identity/v3/users/2d2ff3aaf6c246dc98bbd4cd488bfd64 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:22.456786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:22.458979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 120/238] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Oct 12 12:18:22 2022] GET /identity/v3/role_assignments?user.id=35bb6e05f3074030bc1ca510baffdebb&scope.system=all&role.id=31deee9228fc4ecd930c156b89edb5c6 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:22.526856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:22.572878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e7497-457c-428a-a526-ab048ce350f4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:22.609778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cbbb4-986c-4045-97ac-e39796b0c29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:22.610540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cbbb4-986c-4045-97ac-e39796b0c29f None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:22.610900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cbbb4-986c-4045-97ac-e39796b0c29f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:22.611176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cbbb4-986c-4045-97ac-e39796b0c29f None admin] PATH_INFO: `/v3/system/users/2d2ff3aaf6c246dc98bbd4cd488bfd64/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:22.612719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cbbb4-986c-4045-97ac-e39796b0c29f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2d2ff3aaf6c246dc98bbd4cd488bfd64, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:22.614208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20cbbb4-986c-4045-97ac-e39796b0c29f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:22.629359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 119/239] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Oct 12 12:18:22 2022] PUT /identity/v3/system/users/2d2ff3aaf6c246dc98bbd4cd488bfd64/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:18:24.486536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e01c75-018b-4f6e-83ae-da278fd6b193 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:24.487047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e01c75-018b-4f6e-83ae-da278fd6b193 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:24.487047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e01c75-018b-4f6e-83ae-da278fd6b193 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:24.488453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 121/240] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:24.494783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8580e3-c8ee-4f84-8141-17419cbb4ab7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:24.495312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8580e3-c8ee-4f84-8141-17419cbb4ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:24.495312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8580e3-c8ee-4f84-8141-17419cbb4ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:24.515176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1b8580e3-c8ee-4f84-8141-17419cbb4ab7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:24.516829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1b8580e3-c8ee-4f84-8141-17419cbb4ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:24.584392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 120/241] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:24.613615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d213ee-e2dd-4f10-b575-591500e28b4d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:24.613897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d213ee-e2dd-4f10-b575-591500e28b4d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:24.614175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d213ee-e2dd-4f10-b575-591500e28b4d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:24.615649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 122/242] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:24.620556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9d82-135d-47cb-bf1a-9f6b1253f9a1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:24.620848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9d82-135d-47cb-bf1a-9f6b1253f9a1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:24.621122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe9d82-135d-47cb-bf1a-9f6b1253f9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:24.639766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-edfe9d82-135d-47cb-bf1a-9f6b1253f9a1 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:24.641444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-edfe9d82-135d-47cb-bf1a-9f6b1253f9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:24.698229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:24.717764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 121/243] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:24.775800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd1ef-d564-4318-8cf5-c2b1bb247a53 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:24.776068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd1ef-d564-4318-8cf5-c2b1bb247a53 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:24.776280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28dd1ef-d564-4318-8cf5-c2b1bb247a53 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:24.777804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 123/244] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:24.782837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:24.783126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:24.783403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:24.801257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:24.802460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:24.851360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dd1ef-d564-4318-8cf5-c2b1bb247a53 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:24.872891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 122/245] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:24.941615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:24.964309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28dd1ef-d564-4318-8cf5-c2b1bb247a53 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:24.984080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:24.998990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:24.999808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.000876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.001398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.002662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.004281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.006158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 124/246] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:25.011517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.095080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbcebd-0889-4a7f-8481-1a4313dd4349 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.111489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:25.118470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b4842-58b0-4e30-a64a-19461bdb34e0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.132656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.133292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.133608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.133883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.144741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:25.150404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:25.151318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:25.152138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 123/247] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:25.152823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.157701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.158307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.158588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.158874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] PATH_INFO: `/v3/groups` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.159475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.160931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.189347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 125/248] 10.222.0.17 () {60 vars in 1134 bytes} [Wed Oct 12 12:18:25 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:18:25.189347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.190421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:25.274167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9eb40a-cebc-4179-b007-c1729c63e3e2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.302318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf4330-d614-4a88-9c2a-3fe91efe5a69 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.314726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.315380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.315648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.315913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.328332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:25.333503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:25.334420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:25.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 124/249] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:25.342028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.343463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.344119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.344396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.344865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.345549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:25.345785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.347175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.356736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 126/250] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:25.362767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.383123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:25.447786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:25.456933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069f0942-6d24-4169-9070-90ff7c0fe2d4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.486681 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca4306-0280-4ff5-b2cb-b6bed5247852 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.507029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.507938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.508221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.508493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.509178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:25.509524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.511393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.520880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 125/251] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:25.526452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.527842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.528470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.528743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.529007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] PATH_INFO: `/v3/users/35bb6e05f3074030bc1ca510baffdebb` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.530183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] RBAC: Authorizing `identity:get_user(user_id=35bb6e05f3074030bc1ca510baffdebb)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.532288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.534693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 127/252] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:25 2022] GET /identity/v3/users/35bb6e05f3074030bc1ca510baffdebb => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:25.539993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.647537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:25.653986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da87978-03e7-4b4a-8de5-195f9911e020 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.675962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da89a7-f8bb-4c17-8c37-152f82dd0ef5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.699931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.700563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.700838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.701103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] PATH_INFO: `/v3/users/2d2ff3aaf6c246dc98bbd4cd488bfd64` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.702275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] RBAC: Authorizing `identity:get_user(user_id=2d2ff3aaf6c246dc98bbd4cd488bfd64)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.704190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.706189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 126/253] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:25 2022] GET /identity/v3/users/2d2ff3aaf6c246dc98bbd4cd488bfd64 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:25.712194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:25.721369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8a485c-2499-46cd-8a08-fb580ba1bfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.722346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a485c-2499-46cd-8a08-fb580ba1bfeb None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.722824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a485c-2499-46cd-8a08-fb580ba1bfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.723298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8a485c-2499-46cd-8a08-fb580ba1bfeb None admin] PATH_INFO: `/v3/system/users/35bb6e05f3074030bc1ca510baffdebb/roles/31deee9228fc4ecd930c156b89edb5c6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.729447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a485c-2499-46cd-8a08-fb580ba1bfeb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=35bb6e05f3074030bc1ca510baffdebb, role_id=31deee9228fc4ecd930c156b89edb5c6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.729447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8a485c-2499-46cd-8a08-fb580ba1bfeb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.748945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 128/254] 10.222.0.17 () {58 vars in 1334 bytes} [Wed Oct 12 12:18:25 2022] PUT /identity/v3/system/users/35bb6e05f3074030bc1ca510baffdebb/roles/31deee9228fc4ecd930c156b89edb5c6 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:18:25.849342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac48a5-0b82-4c49-a7e6-f6973adbcf81 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:25.887385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:25.888430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:25.888932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:25.889441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:25.890385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] RBAC: Adding query filter params (role.id=0e327db8aba54600aac0165be40f349b, user.id=2d2ff3aaf6c246dc98bbd4cd488bfd64, scope.system=all) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:25.891467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:25.893221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308324f4-ab1e-4983-9116-dc0f59c1636b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:25.908936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 127/255] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Oct 12 12:18:25 2022] GET /identity/v3/role_assignments?user.id=2d2ff3aaf6c246dc98bbd4cd488bfd64&scope.system=all&role.id=0e327db8aba54600aac0165be40f349b => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:27.697625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478e52f-421d-4651-8333-8171a30bd6a1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:27.698616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478e52f-421d-4651-8333-8171a30bd6a1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:27.699229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b478e52f-421d-4651-8333-8171a30bd6a1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:27.701033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 129/256] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:27.706412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff179d6-072d-4b26-acea-9028fdd2e01f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:27.707252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff179d6-072d-4b26-acea-9028fdd2e01f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:27.707723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff179d6-072d-4b26-acea-9028fdd2e01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:27.732406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3ff179d6-072d-4b26-acea-9028fdd2e01f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:27.733950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3ff179d6-072d-4b26-acea-9028fdd2e01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:27.761828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:27.810602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 128/257] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:18:28.095318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0436b1-e8fa-4a9d-961f-1632961e3193 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.095710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0436b1-e8fa-4a9d-961f-1632961e3193 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.095904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0436b1-e8fa-4a9d-961f-1632961e3193 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.097270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 130/258] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:28.105455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.105747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.106021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.136047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:28.137258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:28.148852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 129/259] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 12 12:18:28.334892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0436b1-e8fa-4a9d-961f-1632961e3193 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.342175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:28.354601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0436b1-e8fa-4a9d-961f-1632961e3193 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.365396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.366314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.366791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.367257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.380474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:28.389871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:28.390826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:28.391642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:28.394320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 131/260] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:28.400407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.411142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2faa47-4415-4368-b671-ac6b9149deab None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.418314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.419178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.419656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.420121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.421011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:28.421589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.424595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:18:28.424595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:18:28.424595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:18:28.425912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.439148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 130/261] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:28.445071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.455610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f509e87-28ba-4bed-b322-c4d2b04c9f08 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.463387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.464315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.464792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.465273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.473526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.475629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:18:28.475629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:18:28.475629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:18:28.477228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.483344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 132/262] 10.222.0.17 () {56 vars in 1201 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:28.490579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.501649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6415b60e-65cf-4606-b5bd-62f4391abc6c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.508971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.509894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.510373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.510837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.518118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.520221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.522247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 131/263] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:28.527904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.545330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edecec21-43b8-42e4-a8d8-517ce6654767 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.550919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.551909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.552391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.552871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5/groups/4dbc647256d3445c9eb54efd5098219f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.555100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e403631cd6479489b75c17c38466d5, group_id=4dbc647256d3445c9eb54efd5098219f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.556373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:18:28.556373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:18:28.556373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:18:28.558837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.586910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 133/264] 10.222.0.17 () {58 vars in 1463 bytes} [Wed Oct 12 12:18:28 2022] PUT /identity/v3/projects/43e403631cd6479489b75c17c38466d5/groups/4dbc647256d3445c9eb54efd5098219f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:28.638839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.655075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4b0e2c-632d-40ae-8e58-ebb530c799a5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.730912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e314fe0c-d042-4287-a559-e14dafea3198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.731526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314fe0c-d042-4287-a559-e14dafea3198 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.731814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314fe0c-d042-4287-a559-e14dafea3198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.732085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e314fe0c-d042-4287-a559-e14dafea3198 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.748950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e314fe0c-d042-4287-a559-e14dafea3198 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e314fe0c-d042-4287-a559-e14dafea3198 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:28.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:28.755318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:28.756120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 132/265] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 117 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:28.762742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.770621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716339e4-0377-41c1-a21c-db19a1a354db None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.770906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716339e4-0377-41c1-a21c-db19a1a354db None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.771179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716339e4-0377-41c1-a21c-db19a1a354db None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.772687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 133/266] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:18:28.778616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.779139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.779543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.784140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc32cb-ce2b-4ab7-8581-4ed63d931e16 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.798491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.799297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.799706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.800072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.800897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:28.801632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.807790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:18:28.807790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:18:28.807790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:18:28.808925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.813605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:28.818737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:28.824422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 134/267] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:28.830681 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.838592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 134/268] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:28 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 12:18:28.850238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ab74c5-56c6-4e44-8e49-d5cde543047a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.855244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.855898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.856228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.856510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.857819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.859949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.861754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 135/269] 10.222.0.17 () {56 vars in 1201 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:28.867518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.880756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd667-34d4-4607-b240-068a0982147f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.889103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dfccdc-dc82-43ea-92e0-344dd571efdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.889845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfccdc-dc82-43ea-92e0-344dd571efdc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.890171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfccdc-dc82-43ea-92e0-344dd571efdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.890501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfccdc-dc82-43ea-92e0-344dd571efdc admin admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.898225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfccdc-dc82-43ea-92e0-344dd571efdc admin admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.900532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dfccdc-dc82-43ea-92e0-344dd571efdc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.902395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 135/270] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Oct 12 12:18:28 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:28.908783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:28.919201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797169d8-42bc-4523-9f59-ed10fd019b0f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:28.930239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5fcd9f-ceaf-4d7d-83a4-78bf5821933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:28.930898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fcd9f-ceaf-4d7d-83a4-78bf5821933c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:28.931227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fcd9f-ceaf-4d7d-83a4-78bf5821933c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:28.931505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5fcd9f-ceaf-4d7d-83a4-78bf5821933c admin admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5/groups/4dbc647256d3445c9eb54efd5098219f/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:28.933843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fcd9f-ceaf-4d7d-83a4-78bf5821933c admin admin] RBAC: Authorizing `identity:create_grant(project_id=43e403631cd6479489b75c17c38466d5, group_id=4dbc647256d3445c9eb54efd5098219f, role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:28.936715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5fcd9f-ceaf-4d7d-83a4-78bf5821933c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:28.964985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 136/271] 10.222.0.17 () {58 vars in 1463 bytes} [Wed Oct 12 12:18:28 2022] PUT /identity/v3/projects/43e403631cd6479489b75c17c38466d5/groups/4dbc647256d3445c9eb54efd5098219f/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:29.156010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edc203-0b6b-479d-9b5e-322d59ae60ea None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.156968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edc203-0b6b-479d-9b5e-322d59ae60ea None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.157138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edc203-0b6b-479d-9b5e-322d59ae60ea None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.158589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 136/272] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:29.171148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.171467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.171680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.189765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edc203-0b6b-479d-9b5e-322d59ae60ea None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:29.191491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:29.202502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:29.353009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edc203-0b6b-479d-9b5e-322d59ae60ea None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:29.362692 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:29.363335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.363615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.363885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.382579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 137/273] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:29.385684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:29.390849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:29.391851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:29.392631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:29.396619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 137/274] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:18:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:29.398028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:29.413356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8c7788-bc74-43f5-8ed1-2af6c7495826 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:29.420764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:29.421425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.421709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.422535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.423224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] RBAC: Adding query filter params (name=member) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:29.423529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:29.425164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:29.439966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 138/275] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:18:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:29.446570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:29.458172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc65b3d-56ba-435c-8e65-bc11129c832b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:29.464410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:29.465045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.465340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.465616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.466749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:29.467769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:18:29.467769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:18:29.467769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:18:29.469102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:29.471088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 138/276] 10.222.0.17 () {56 vars in 1201 bytes} [Wed Oct 12 12:18:29 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:29.476203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:29.490422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d94114-b1c8-43f2-813c-a10f521a9e9c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:29.499456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:29.500742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.500961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.501237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.502424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:29.504501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:29.507136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 139/277] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Oct 12 12:18:29 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:29.511724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:29.523697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b4931-f255-408d-b78d-59298f71e863 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:29.528237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:29.528957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.529243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.529549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/groups/4dbc647256d3445c9eb54efd5098219f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.534562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=731ddd9bfa3c4ac7947beb04adaef28c, group_id=4dbc647256d3445c9eb54efd5098219f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:29.535592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:18:29.535592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:18:29.535592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:18:29.537616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d04217-0747-44e0-b42f-24fc4f4efcb5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:29.561775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 139/278] 10.222.0.17 () {58 vars in 1463 bytes} [Wed Oct 12 12:18:29 2022] PUT /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/groups/4dbc647256d3445c9eb54efd5098219f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:29.779287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:29.855431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:29.911658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b18188e-9a6a-4d93-8481-82d355f1c250 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:29.914186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53628-5c0a-48d5-bce7-6a50b7b2482e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.914631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53628-5c0a-48d5-bce7-6a50b7b2482e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.914716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d53628-5c0a-48d5-bce7-6a50b7b2482e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.916390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 140/279] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:29.925677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.925959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.926236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:29.935928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:29.959048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:29.960254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:29.994787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:29.995422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:29.995695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:29.995964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.021263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:30.026635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:30.027735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 12 12:18:30.028753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:30.031308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 140/280] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:30.036439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.075241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 141/281] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:29 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 12:18:30.169118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b60a5a0-34b0-433d-a496-f7bfef30acf1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.195232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:30.216697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.217678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.218157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.218606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.219506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:30.220041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.221831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.240085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 141/282] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:30.246700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.362081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488d6d-d7d0-451e-9669-e63884c60f21 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.404896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.405863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.406515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.407019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] PATH_INFO: `/v3/users/35bb6e05f3074030bc1ca510baffdebb` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.408355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] RBAC: Authorizing `identity:get_user(user_id=35bb6e05f3074030bc1ca510baffdebb)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.410025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.412138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 142/283] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/users/35bb6e05f3074030bc1ca510baffdebb => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:30.422390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.542663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705ed39a-629a-495d-ae12-90eb0b063067 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.559159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.589338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.589338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.589338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.589338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.589756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] RBAC: Adding query filter params (role.id=31deee9228fc4ecd930c156b89edb5c6, user.id=35bb6e05f3074030bc1ca510baffdebb, scope.system=all) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:30.589756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.589756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.591249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec7c9e-a3ef-4def-a8be-f231676f67ad None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.608299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 142/284] 10.222.0.17 () {56 vars in 1308 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/role_assignments?user.id=35bb6e05f3074030bc1ca510baffdebb&scope.system=all&role.id=31deee9228fc4ecd930c156b89edb5c6 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:30.618263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.619270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.619749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.620215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.638202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:30.644545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:30.645554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:30.646352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:30.648871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 143/285] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 90 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:30.655654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.676405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c1455-0f7e-40a5-98d4-a89e6d7472b4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.687253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.688578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.689026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.689300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.690028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:30.690328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.692018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.709216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 143/286] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:30.714934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.724196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:30.731256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec7434-5278-42d6-819c-ba77e02a2847 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.741724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.742690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.743170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.743666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.750223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.752518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.755268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 144/287] 10.222.0.17 () {56 vars in 1201 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:30.761018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.774791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abd302a-8064-44dd-8359-bb6c0295498f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.783241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27d370-2a1c-4ab4-96c9-d4a98551f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.783862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27d370-2a1c-4ab4-96c9-d4a98551f2af admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.784139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27d370-2a1c-4ab4-96c9-d4a98551f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.784406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27d370-2a1c-4ab4-96c9-d4a98551f2af admin admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.790253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27d370-2a1c-4ab4-96c9-d4a98551f2af admin admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.792426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e27d370-2a1c-4ab4-96c9-d4a98551f2af admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.794517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 144/288] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Oct 12 12:18:30 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:30.801194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:30.811637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829ff91-93c2-409a-aff4-fa43f30a7c7c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:30.821865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:30.822861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.823341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.823975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/groups/4dbc647256d3445c9eb54efd5098219f/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.827936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=731ddd9bfa3c4ac7947beb04adaef28c, group_id=4dbc647256d3445c9eb54efd5098219f, role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:30.830981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:30.842302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39e7f5-0378-4c08-bae8-529e7e770957 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.842610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39e7f5-0378-4c08-bae8-529e7e770957 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.842836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39e7f5-0378-4c08-bae8-529e7e770957 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.845048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 145/289] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:30.850929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:30.851212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:30.851481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:30.857380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 145/290] 10.222.0.17 () {58 vars in 1463 bytes} [Wed Oct 12 12:18:30 2022] PUT /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c/groups/4dbc647256d3445c9eb54efd5098219f/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:30.881768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:30.883883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:30.926008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:30.967385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 146/291] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:30 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 12 12:18:31.290819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:31.302554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036dcdcb-67b0-4595-b870-2d3b7edddd2a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:31.313475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:31.314320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.314746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.315164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:31.329464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:31.334580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:31.335647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:31.336646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:31.341072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 146/292] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:18:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:31.347295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:31.356516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f0f0c0-42a5-430b-b1f4-6caa7b0e7633 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:31.365447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:31.366076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.366353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.366624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:31.367308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:31.367610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:31.369263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:31.385008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 147/293] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:18:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:31.390304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:31.401466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3b67e3-e79e-4edd-bbb1-666c19682c8c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:31.413539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:31.414426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.414879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.415332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] PATH_INFO: `/v3/groups/1c2d2c31ab064571828c39d90ffe8bbf` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:31.433963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] RBAC: Authorizing `identity:get_group(group_id=1c2d2c31ab064571828c39d90ffe8bbf)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:31.436192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:31.439229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 147/294] 10.222.0.17 () {56 vars in 1201 bytes} [Wed Oct 12 12:18:31 2022] GET /identity/v3/groups/1c2d2c31ab064571828c39d90ffe8bbf => generated 229 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:31.444435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:31.459185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702483d4-dcc1-4bc4-b09a-2e9935ca55ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:31.469339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37936633-34f7-4bfd-bcb3-1e903bcd384e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:31.469987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936633-34f7-4bfd-bcb3-1e903bcd384e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.470271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936633-34f7-4bfd-bcb3-1e903bcd384e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.470550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37936633-34f7-4bfd-bcb3-1e903bcd384e admin admin] PATH_INFO: `/v3/projects/22af725251c64418b4185f46ec322cf9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:31.471688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37936633-34f7-4bfd-bcb3-1e903bcd384e admin admin] RBAC: Authorizing `identity:get_project(project_id=22af725251c64418b4185f46ec322cf9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:31.473548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37936633-34f7-4bfd-bcb3-1e903bcd384e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:31.475469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 148/295] 10.222.0.17 () {56 vars in 1207 bytes} [Wed Oct 12 12:18:31 2022] GET /identity/v3/projects/22af725251c64418b4185f46ec322cf9 => generated 344 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:31.480761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:31.491425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f6beb-b83e-49d8-bf5f-e69f3e510d28 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:31.508782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c5c6c-e25c-47fb-999d-aa4e61629ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:31.509838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c5c6c-e25c-47fb-999d-aa4e61629ebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.510320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c5c6c-e25c-47fb-999d-aa4e61629ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.510793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c5c6c-e25c-47fb-999d-aa4e61629ebb admin admin] PATH_INFO: `/v3/projects/22af725251c64418b4185f46ec322cf9/groups/1c2d2c31ab064571828c39d90ffe8bbf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:31.518663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c5c6c-e25c-47fb-999d-aa4e61629ebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=22af725251c64418b4185f46ec322cf9, group_id=1c2d2c31ab064571828c39d90ffe8bbf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:31.521631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c5c6c-e25c-47fb-999d-aa4e61629ebb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:31.544456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 148/296] 10.222.0.17 () {58 vars in 1463 bytes} [Wed Oct 12 12:18:31 2022] PUT /identity/v3/projects/22af725251c64418b4185f46ec322cf9/groups/1c2d2c31ab064571828c39d90ffe8bbf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:31.970091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99666fc1-4c12-4c17-9402-f3262cb5d7c8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.970573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99666fc1-4c12-4c17-9402-f3262cb5d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.970648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99666fc1-4c12-4c17-9402-f3262cb5d7c8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:31.972237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 149/297] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:31.977467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78fc3-428f-49d1-9fee-29b51c79eca6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:31.977990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78fc3-428f-49d1-9fee-29b51c79eca6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:31.978480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd78fc3-428f-49d1-9fee-29b51c79eca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:32.002832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9bd78fc3-428f-49d1-9fee-29b51c79eca6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:32.004038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9bd78fc3-428f-49d1-9fee-29b51c79eca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:32.093733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 149/298] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:32.268023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732778a2-f09a-4a87-8858-119f9eca4362 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:32.268307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732778a2-f09a-4a87-8858-119f9eca4362 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:32.268855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732778a2-f09a-4a87-8858-119f9eca4362 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:32.270238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 150/299] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:32.275743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:32.276064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:32.276343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:32.302376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:32.303591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:32.391976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 150/300] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:32.506603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732778a2-f09a-4a87-8858-119f9eca4362 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:32.638649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732778a2-f09a-4a87-8858-119f9eca4362 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:32.682045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:32.682676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:32.682953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:32.683225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:32.699855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:32.705410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:32.706310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:32.707116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 151/301] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:32.712771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:32.745289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:32.781663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:32.821682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:32.888188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41e560a-2a86-4c0d-9b8e-8d63a451096c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:32.936835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:32.937890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:32.938105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:32.938423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:32.939208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:32.939681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:32.941833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:32.954747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 151/302] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:32.960778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:32.975363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad72c79-3b2c-4c07-89b9-32549760d77a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.039065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.039807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.040084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.040352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.071597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:33.074822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:33.077437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:33.078947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:33.079738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 152/303] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:18:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:33.086347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:33.103593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa67af-c220-4793-92c7-04dfe922ff46 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.152677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d144de70-eac2-4550-9a70-f4622a112f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.154043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144de70-eac2-4550-9a70-f4622a112f34 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.154323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144de70-eac2-4550-9a70-f4622a112f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.154625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d144de70-eac2-4550-9a70-f4622a112f34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.191140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:33.217997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7e415-3673-4292-8bae-e667c439bff1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.249004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144de70-eac2-4550-9a70-f4622a112f34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:33.250886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d144de70-eac2-4550-9a70-f4622a112f34 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:33.265040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.266454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.266653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.266943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.267594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:33.267899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:33.269296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816777fc-5551-40dd-90d1-02fe24582b4f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:33.280039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 153/304] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:33.282058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c0d7-43d0-4ca8-8d6b-abb2d1e09847 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.282348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c0d7-43d0-4ca8-8d6b-abb2d1e09847 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.282698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c0d7-43d0-4ca8-8d6b-abb2d1e09847 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.283913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 154/305] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:33.288514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c0d7-43d0-4ca8-8d6b-abb2d1e09847 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:33.346010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:33.376500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 152/306] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 12 12:18:33.378201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.378709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.379188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.399306 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:33.400809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:33.409158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c0d7-43d0-4ca8-8d6b-abb2d1e09847 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.451807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.452707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.453674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.454087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.466747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:33.491479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 153/307] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:33.492681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:33.545213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:33.548102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:33.665748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 155/308] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:33.671156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:33.700806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b4df89-95c8-493e-b08e-ac55822a458d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.725656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:33.749193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:33.777711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.778367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.778648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.778925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.780955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:33.783001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:33.787804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 154/309] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:33 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:33.793058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:33.859085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f850f-ee85-429a-bf45-04f9cbd16111 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.914726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.915707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.916187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.916677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] PATH_INFO: `/v3/projects/43e403631cd6479489b75c17c38466d5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.918069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] RBAC: Authorizing `identity:get_project(project_id=43e403631cd6479489b75c17c38466d5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:33.919720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:33.922672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 156/310] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:33 2022] GET /identity/v3/projects/43e403631cd6479489b75c17c38466d5 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:33.923679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:33.952650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a965ce46-67e0-4865-8c74-7e43f6076640 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:33.997255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:33.997852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:33.998129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:33.998420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:33.999521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:34.001086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:34.003004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 155/311] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Oct 12 12:18:33 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:34.003782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.068543 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b02c4a-7cb9-47a0-ad36-1da0686bcadc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.069738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:34.081722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:34.115139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc785e5-0e1a-4616-b697-9a0f34b0af0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:34.115781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc785e5-0e1a-4616-b697-9a0f34b0af0b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.116056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc785e5-0e1a-4616-b697-9a0f34b0af0b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.116348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc785e5-0e1a-4616-b697-9a0f34b0af0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.132581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc785e5-0e1a-4616-b697-9a0f34b0af0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cfc785e5-0e1a-4616-b697-9a0f34b0af0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:34.138415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:34.139308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 12 12:18:34.140091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:34.142107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 157/312] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:34.143243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb619f-0f29-4734-ac3a-70de7c78c7e5 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.143522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb619f-0f29-4734-ac3a-70de7c78c7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.143698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c41ad-b80c-4275-b788-147a54ddc464 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.143798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb619f-0f29-4734-ac3a-70de7c78c7e5 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.145380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 158/313] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:34.146138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb619f-0f29-4734-ac3a-70de7c78c7e5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.184282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:34.184914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.185191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.185492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.186588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:34.188766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:34.190373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 156/314] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:34.191140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.283707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb619f-0f29-4734-ac3a-70de7c78c7e5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.322583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6442c2f-e2f4-4719-b9d0-479b113f877a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.324456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:34.325061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.326230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.326230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.326431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] RBAC: Adding query filter params (group.id=4dbc647256d3445c9eb54efd5098219f, role.id=0e327db8aba54600aac0165be40f349b, scope.project.id=43e403631cd6479489b75c17c38466d5) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:34.326663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:34.328172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fe3e0a-f087-46a6-9d53-7b05b2ee83be None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:34.336575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 159/315] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/role_assignments?group.id=4dbc647256d3445c9eb54efd5098219f&scope.project.id=43e403631cd6479489b75c17c38466d5&role.id=0e327db8aba54600aac0165be40f349b => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:34.338417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.338417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.338417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.343991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:34.362482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:34.365003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:34.376366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:34.379304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.379839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.380287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.381225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:34.381751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:34.383392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:34.394262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 157/316] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:34.395427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.398710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:34.475830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 160/317] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:34.481270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.532652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e06d4bb-ab35-4b16-9579-429a93cffd62 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.584619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:34.584969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.585239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.585543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.586281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] RBAC: Adding query filter params (group.id=4dbc647256d3445c9eb54efd5098219f, role.id=2b9a144dd3114f13a8fa1006f585fb12, scope.project.id=43e403631cd6479489b75c17c38466d5) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:34.586591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:34.588057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb8913a-5db0-46a6-9012-25cbd3c75857 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:34.594866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 158/318] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/role_assignments?group.id=4dbc647256d3445c9eb54efd5098219f&scope.project.id=43e403631cd6479489b75c17c38466d5&role.id=2b9a144dd3114f13a8fa1006f585fb12 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:34.596103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7ac3-c867-40f1-8821-0bd046893b76 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.596390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7ac3-c867-40f1-8821-0bd046893b76 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.596729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e7ac3-c867-40f1-8821-0bd046893b76 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.601100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 159/319] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:34.607138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.607486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.607688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.622931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012a59c-cffd-44e6-b565-a858ea5f8eec None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.636806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:34.638641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:34.669580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:34.670600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:34.671134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:34.671667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:34.688677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:34.727463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:34.740637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 160/320] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:34.785361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:34.787428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:34.849822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.901990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 161/321] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:34.916021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:34.987618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db62f56-64c9-446d-9a4a-86b590e99062 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:34.988258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.029709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.031310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.031310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.031310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.038977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72480c23-74cd-4947-91a5-6290ac4bf783 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.047784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:35.052701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:35.053685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:35.054502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 161/322] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:35.058113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.079189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.079822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.080104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.080373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.081590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.083038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.084672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 162/323] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:34 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:35.086476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.096304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.184291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86475b8-c487-493d-a576-8ab5a9b3f73e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.202722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b814e98-9e34-4336-b8e6-b93ef2112dd3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.221874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.222542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.222821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.223102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.233786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:35.239316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:35.240267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:35.241292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 162/324] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:35.242668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.243605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.247787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.248864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.249332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.249641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.250557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:35.250856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.252674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.270603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 163/325] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:35.273360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.347386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.367056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9071803-55ac-4c70-bf5e-e68a231de0d3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.393850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1acfaf-56e3-4e2f-a694-1b8236ab3a85 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.407104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.407794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.408080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.408819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.409682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.411213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.412712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 163/326] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:35.414213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.442772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.443385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.443668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.443941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.444607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:35.444911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.446372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.455923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 164/327] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:35.456770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.524385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.532106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a88a95-336f-49dd-bf40-5c864209a1ce None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.568900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.569540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.569815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.570376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.577090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682833ac-9634-4ab7-928b-67263ac46614 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.598894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.621867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.623013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.623734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.624156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.625865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] RBAC: Adding query filter params (group.id=4dbc647256d3445c9eb54efd5098219f, role.id=0e327db8aba54600aac0165be40f349b, scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:35.626305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.627892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.636587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 165/328] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/role_assignments?group.id=4dbc647256d3445c9eb54efd5098219f&scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c&role.id=0e327db8aba54600aac0165be40f349b => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:35.636741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.664510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.668719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.767991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a32160-0995-4c0b-b502-c699632b023e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.788840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.794663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 164/329] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:35.795191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:35.811411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.812029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.812354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.812726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.861026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:35.897504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.899199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.915975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55622d-02cb-4f0c-ac02-9c7d5e293cda None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:35.959980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:35.960635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:35.960916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:35.961476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] PATH_INFO: `/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:35.963935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] RBAC: Authorizing `identity:get_project(project_id=731ddd9bfa3c4ac7947beb04adaef28c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:35.965883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:35.968005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 165/330] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/projects/731ddd9bfa3c4ac7947beb04adaef28c => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:35.974321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:36.022449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 166/331] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:36.028081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:36.046064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:36.082747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b1dcc-bb47-45fe-9edc-acaeb2cfa749 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:36.105111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:36.115895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:36.116455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:36.116743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:36.117015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] PATH_INFO: `/v3/groups/4dbc647256d3445c9eb54efd5098219f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:36.118148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] RBAC: Authorizing `identity:get_group(group_id=4dbc647256d3445c9eb54efd5098219f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:36.119602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:36.121513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 166/332] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Oct 12 12:18:35 2022] GET /identity/v3/groups/4dbc647256d3445c9eb54efd5098219f => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:36.126974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b403fae-24e5-4e63-9a43-dbfdd3ad15b8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:36.127783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:36.160677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:36.161297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:36.161605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:36.161877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] PATH_INFO: `/v3/projects/22af725251c64418b4185f46ec322cf9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:36.162987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] RBAC: Authorizing `identity:get_project(project_id=22af725251c64418b4185f46ec322cf9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:36.164445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:36.166135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 167/333] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:18:36 2022] GET /identity/v3/projects/22af725251c64418b4185f46ec322cf9 => generated 344 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:36.171410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:36.225981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab626e6-b849-4322-a9ae-1a310b258202 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:36.259877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:36.262645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:36.263260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:36.263632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:36.263814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:36.264552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] RBAC: Adding query filter params (group.id=4dbc647256d3445c9eb54efd5098219f, role.id=2b9a144dd3114f13a8fa1006f585fb12, scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:36.264861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:36.267230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:36.273420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 167/334] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Oct 12 12:18:36 2022] GET /identity/v3/role_assignments?group.id=4dbc647256d3445c9eb54efd5098219f&scope.project.id=731ddd9bfa3c4ac7947beb04adaef28c&role.id=2b9a144dd3114f13a8fa1006f585fb12 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:36.277429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a05eec-2bdf-4491-9fbe-5550b6f2f507 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:36.316740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dbfe0e-dbd0-4e64-b948-7d45d6c045a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:36.317410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbfe0e-dbd0-4e64-b948-7d45d6c045a6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:36.317690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbfe0e-dbd0-4e64-b948-7d45d6c045a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:36.317960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dbfe0e-dbd0-4e64-b948-7d45d6c045a6 None admin] PATH_INFO: `/v3/groups/1c2d2c31ab064571828c39d90ffe8bbf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:36.319088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbfe0e-dbd0-4e64-b948-7d45d6c045a6 None admin] RBAC: Authorizing `identity:get_group(group_id=1c2d2c31ab064571828c39d90ffe8bbf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:36.320555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dbfe0e-dbd0-4e64-b948-7d45d6c045a6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:36.322225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 168/335] 10.222.0.17 () {56 vars in 1180 bytes} [Wed Oct 12 12:18:36 2022] GET /identity/v3/groups/1c2d2c31ab064571828c39d90ffe8bbf => generated 229 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:36.328750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:36.390218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:36.436461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2e42ad-d579-4e90-bd32-14e6da2ee54e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:36.472719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:36.473348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:36.473637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:36.473908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:36.474639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] RBAC: Adding query filter params (group.id=1c2d2c31ab064571828c39d90ffe8bbf, role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=22af725251c64418b4185f46ec322cf9) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:36.475076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:36.477733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5fbf78-923e-4d6c-a8f6-2487255fd067 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:36.487512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 168/336] 10.222.0.17 () {56 vars in 1376 bytes} [Wed Oct 12 12:18:36 2022] GET /identity/v3/role_assignments?group.id=1c2d2c31ab064571828c39d90ffe8bbf&scope.project.id=22af725251c64418b4185f46ec322cf9&role.id=2e857246137f4d06b40cdabedf42881f => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:39.113650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bbc85-4ff4-47c2-8dc3-99ec263f2ec2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.114694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bbc85-4ff4-47c2-8dc3-99ec263f2ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.115248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bbc85-4ff4-47c2-8dc3-99ec263f2ec2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.117139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 169/337] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:39.122284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90274-17ff-4fc7-8293-5c2ebaf0d2ee None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.123104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90274-17ff-4fc7-8293-5c2ebaf0d2ee None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.123604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90274-17ff-4fc7-8293-5c2ebaf0d2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.154839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-18b90274-17ff-4fc7-8293-5c2ebaf0d2ee None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:39.156527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-18b90274-17ff-4fc7-8293-5c2ebaf0d2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:39.251922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 169/338] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:39.318546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4cbe5-d139-4ecd-b3fb-252ccc1e12c6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.319168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4cbe5-d139-4ecd-b3fb-252ccc1e12c6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.319665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4cbe5-d139-4ecd-b3fb-252ccc1e12c6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.321599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 170/339] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:39.326685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b13fba-759a-45c9-b8fe-5697bbb0afe5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.327217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b13fba-759a-45c9-b8fe-5697bbb0afe5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.327753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b13fba-759a-45c9-b8fe-5697bbb0afe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.355517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-29b13fba-759a-45c9-b8fe-5697bbb0afe5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:39.357059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-29b13fba-759a-45c9-b8fe-5697bbb0afe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:39.425675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:39.455074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 170/340] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:39.536126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca0d2a-169d-4af0-b8bb-1cfa40ca661c None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.536773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca0d2a-169d-4af0-b8bb-1cfa40ca661c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.537255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ca0d2a-169d-4af0-b8bb-1cfa40ca661c None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.539136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 171/341] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:39.545433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.545980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.547022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.573289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:39.573289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:39.574760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca0d2a-169d-4af0-b8bb-1cfa40ca661c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:39.706117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 171/342] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:39.751980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ca0d2a-169d-4af0-b8bb-1cfa40ca661c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:39.800736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:39.801379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:39.801655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:39.801921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:39.803762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:39.817130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:39.817130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:39.834033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:39.835200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:39.836182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:39.836182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:39.836182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:39.837904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 172/343] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:39.844004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:39.862499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:39.983993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ecab6-b989-4896-9277-c82ef904a49e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:39.986583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4950f62b-0ee3-49b3-bdb4-65ccf2550326 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.000924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.020971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.022081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.022632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.023070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.023908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:40.024360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.025911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f411f1-82e7-4d00-8da6-8d1774c94df5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.027876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.028503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.028787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.029063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.039825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.041813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.044598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 173/344] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:40.045813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c83b87-db0d-4889-a2ce-9df54cbc7061 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.046271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c83b87-db0d-4889-a2ce-9df54cbc7061 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.047417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c83b87-db0d-4889-a2ce-9df54cbc7061 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.048785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 174/345] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:40.050142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c83b87-db0d-4889-a2ce-9df54cbc7061 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:40.063233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:40.064134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:40.065005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:40.065005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:40.065005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:40.065005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 172/346] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:40.065230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.127637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.164930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c83b87-db0d-4889-a2ce-9df54cbc7061 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.185339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f225d-3086-446c-87d6-7c111b0a7cb7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.205789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.207120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.207562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.207842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.217737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.219352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.221101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.221910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.222018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.222252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.222892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.224300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1c585162-6720-4f1f-988d-e8b231e7f441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:40.235030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:40.236103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:40.236980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:40.236980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:40.236980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:40.236980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 175/347] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:40.238079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.238079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.238079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.261756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 173/348] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:40 2022] POST /identity/v3/users => generated 246 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:40.261756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.262735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:40.263970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:40.288266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.361338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 176/349] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:40.363086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.387846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4480fb4e-c08a-4f3a-8d7a-d4fb19e092f6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.430063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.446682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.447700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.448186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.448683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.449614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:40.451139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.453062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.486468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 174/350] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:40.491671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.507538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59df5d5c-73cc-4fc5-87b0-5e729dfb4e56 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.548650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.549261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.549574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.549842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.551086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:40.551086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.552294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.566966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 177/351] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:40.572268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.615616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.640955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2007de5-d01d-423c-82ad-c7e5c976bfa4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.685985 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.686597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.686873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.687302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.688143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.690320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.694273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e00db38-d9d9-414b-b838-8fe5c3bccafd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.704574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.713980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 175/352] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:40 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:18:40.715015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:40.758405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0aeccf-c8a6-43ff-a149-6d83cb7c583c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.758968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aeccf-c8a6-43ff-a149-6d83cb7c583c None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.759248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aeccf-c8a6-43ff-a149-6d83cb7c583c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.759520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0aeccf-c8a6-43ff-a149-6d83cb7c583c None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.761717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aeccf-c8a6-43ff-a149-6d83cb7c583c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.766501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0aeccf-c8a6-43ff-a149-6d83cb7c583c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.821487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 178/353] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:40 2022] POST /identity/v3/users => generated 248 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:18:40.903197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aee532-5ce9-4e8b-8126-590300ccca83 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:40.942748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01284e-b93b-4bca-be92-752b03a69f84 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.943067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01284e-b93b-4bca-be92-752b03a69f84 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.943312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01284e-b93b-4bca-be92-752b03a69f84 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.944839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 179/354] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:18:40.945440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:40.946238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.946527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.946806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.949199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:40.950546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:40.950834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:40.951116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:40.963779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:40.966275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:40.971821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:40.973146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:40.985923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:40.986884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:40.987725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:40.987725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:40.987725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:40.987725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 176/355] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:40.994974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:41.072258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 180/356] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:41.126767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a438c1-5f88-4ae9-bb60-ac4bb2fb56d6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:41.165703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:41.166361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:41.166643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:41.166916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:41.167557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:41.167863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:41.169239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:41.183075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 177/357] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:41.188509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:41.326995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:41.335815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a3280d-4437-46ba-b0fe-c37c03a3966f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:41.386911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:41.387515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:41.387795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:41.388061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:41.388697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:41.390194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:41.393143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:41.430024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 181/358] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:41 2022] POST /identity/v3/users => generated 249 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:41.450426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:41.514364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9a1cf-2716-4db7-a023-1f6f05a11959 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:41.556844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:41.557490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:41.557772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:41.558043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:41.558728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:41.560108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:41.570224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.sql.core [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 12:18:41.570224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 12:18:41.570224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [parameters: {'id': '5d028eb36dac4a049dc131ee7bb4a1e1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 12:18:41.570224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:18:41.577444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return ref.to_dict() Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:18:41.578521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application yield resource Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise self.value Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application session.commit() Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:18:41.579576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:18:41.580429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:18:41.582070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:18:41.584077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [parameters: {'id': '5d028eb36dac4a049dc131ee7bb4a1e1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:41.585272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 12 12:18:41.586779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:41.587885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 178/359] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Oct 12 12:18:41 2022] POST /identity/v3/roles => generated 144 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:41.587969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:41.743031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a4b8bc-30f5-4a45-9ee4-09c3c6ae2200 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:41.779363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e3218-8b99-4020-8e46-63c9c4a95c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:41.780040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3218-8b99-4020-8e46-63c9c4a95c02 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:41.780315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3218-8b99-4020-8e46-63c9c4a95c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:41.780579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e3218-8b99-4020-8e46-63c9c4a95c02 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:41.791098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e3218-8b99-4020-8e46-63c9c4a95c02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0d5e3218-8b99-4020-8e46-63c9c4a95c02 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:41.796914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:41.798790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:41.800738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 182/360] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:18:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:41.806471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:41.920405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a425f667-1480-4a72-9c9d-9e9c920b20d6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:41.932456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:41.972342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:41.973033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:41.973336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:41.973607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:41.974328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:41.974629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:41.976010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b58443-2c42-40dc-ae54-913f41872f59 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:41.984978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 179/361] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:43.068297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ad29b-282a-493d-9c97-4894c9f4572f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.068820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ad29b-282a-493d-9c97-4894c9f4572f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.068820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ad29b-282a-493d-9c97-4894c9f4572f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.070457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 183/362] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:43.075681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f6fc0-4319-45f8-afa1-e3ad7c7e6768 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.076526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f6fc0-4319-45f8-afa1-e3ad7c7e6768 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.077008 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f6fc0-4319-45f8-afa1-e3ad7c7e6768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.102648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-284f6fc0-4319-45f8-afa1-e3ad7c7e6768 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:43.103841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-284f6fc0-4319-45f8-afa1-e3ad7c7e6768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:43.188334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 180/363] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:43.402797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58ff3-0df7-4397-9ba1-8e308d2e5408 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.402980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58ff3-0df7-4397-9ba1-8e308d2e5408 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.403244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb58ff3-0df7-4397-9ba1-8e308d2e5408 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.404820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 184/364] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:43.410439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.410724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.410998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.438666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:43.439868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:43.491555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb58ff3-0df7-4397-9ba1-8e308d2e5408 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:43.517428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:43.544434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 181/365] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:43.651304 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb58ff3-0df7-4397-9ba1-8e308d2e5408 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:43.700316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516b6dd7-6bc9-41fb-8584-61509031b6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:43.700955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b6dd7-6bc9-41fb-8584-61509031b6df None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.701237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b6dd7-6bc9-41fb-8584-61509031b6df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.701529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516b6dd7-6bc9-41fb-8584-61509031b6df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.721759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516b6dd7-6bc9-41fb-8584-61509031b6df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-516b6dd7-6bc9-41fb-8584-61509031b6df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:43.727214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:43.728221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:43.729026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 185/366] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:43.736744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:43.775259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:43.826094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feea161-df18-4402-ae32-6113b0c3fa6f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.826415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feea161-df18-4402-ae32-6113b0c3fa6f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.826624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feea161-df18-4402-ae32-6113b0c3fa6f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.828646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 186/367] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:18:43.833875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.834151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.834421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.858500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:43.860213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:43.887272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5087988-3aa3-4d96-b095-43688d77f601 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:43.933764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:43.934721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:43.935199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:43.935687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:43.936566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:43.937834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:43.939682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:43.949106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 182/368] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:43.955441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:43.960562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:43.969340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 187/369] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:18:43.969340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.070615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2936-c269-4c2f-b55f-4f11a8748985 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.106350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.106977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.107250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.107515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.118399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.119912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:44.136538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:44.137469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:44.138349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:44.138349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:44.138349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:44.138349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 188/370] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:44.143277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.200191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:44.213875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:44.239269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4211b-fe56-429a-a3b9-c1338bb2f06e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.249003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ddcb8a-eb00-4788-956d-35199e8d0b21 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.284446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.285050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.285346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.285928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.286250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:44.286594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.287963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.294147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.294781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.295061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.295330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.301874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 189/371] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:44.305884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.312439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:44.318102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:44.320279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 183/372] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:44.321625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.433534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd6152-33b5-413b-a8e9-8d2a71301630 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.454094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:44.460119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea351a-f2b5-4495-97d0-ae1980a14884 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.480930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48777f59-6855-4718-826d-befdd166a025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.481592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48777f59-6855-4718-826d-befdd166a025 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.481874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48777f59-6855-4718-826d-befdd166a025 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.482515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48777f59-6855-4718-826d-befdd166a025 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.493445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48777f59-6855-4718-826d-befdd166a025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:44.496910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.497576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.497856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.498126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-48777f59-6855-4718-826d-befdd166a025 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:44.498588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:44.500016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:44.500843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 190/373] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:44.502163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48777f59-6855-4718-826d-befdd166a025 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.512206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.513686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2a5a52e3-c082-49b9-b488-78bc5a9c5294 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:44.527552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:44.528491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 184/374] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5d53-2b9b-4b03-b3e2-9347daaa0a95 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.534125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5d53-2b9b-4b03-b3e2-9347daaa0a95 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.534819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74f5d53-2b9b-4b03-b3e2-9347daaa0a95 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.534819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 185/375] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:44.534819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5d53-2b9b-4b03-b3e2-9347daaa0a95 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.552066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:44.618938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48777f59-6855-4718-826d-befdd166a025 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.645288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74f5d53-2b9b-4b03-b3e2-9347daaa0a95 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.653829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.654440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.654724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.654999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.655728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:44.656220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.657738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.666102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 191/376] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:44.666879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.684806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.685441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.685720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.686000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.686657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:44.687597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.689117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69885665-d621-497e-96d9-2341d6a7b099 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.697531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 186/377] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:44.698759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.699043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.699313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.708035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:44.717181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:44.718446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:44.778629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88067c-aa7a-4fb1-b4c8-fd9cc248a8ef None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.788075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 187/378] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:44.788847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.799278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:44.835950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.836557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.836831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.837100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.838369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:44.838677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.840534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.881713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 192/379] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:44.883034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:44.911162 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fda3fe7-b7f1-4df4-8105-4f1d4466af40 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:44.950349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:44.951124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:44.951401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:44.951671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:44.962493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:44.963919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:44.978060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:44.978969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:44.979818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:44.979818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:44.979818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:44.979818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 188/380] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:44.980040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.003390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.014118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f302c9-5a24-483d-bdcf-fec2f6f8a46f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.039956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.053275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.054144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.054418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.054742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.065821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.067294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f8ed7578-df11-4671-8b19-4cc3d0473352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:45.082327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:45.083649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:45.084506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:45.084506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:45.084506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:45.084506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 193/381] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:45.084722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefd0d5b-55a4-4aa8-b837-dbb513039819 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.084802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe9c1-db83-40b1-a7c0-774788a976d8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.084802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe9c1-db83-40b1-a7c0-774788a976d8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.085206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bbe9c1-db83-40b1-a7c0-774788a976d8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.086420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 194/382] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:45.086937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbe9c1-db83-40b1-a7c0-774788a976d8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.117469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.118097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.118379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.118653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.130571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.132019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:45.146452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:45.147352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:45.148235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:45.148235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:45.148235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:45.152050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 189/383] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:45.152911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.191292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bbe9c1-db83-40b1-a7c0-774788a976d8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.224277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.224869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.225143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.225443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.226069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:45.226364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.227736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.239322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 195/384] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:45.240108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.243383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.255969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc5a85-1e2a-4365-aa14-56b0c9f05403 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.270934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.294126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8e8c9-1050-4199-9b3f-85b6d7ce6b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.294675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8e8c9-1050-4199-9b3f-85b6d7ce6b20 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.294953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8e8c9-1050-4199-9b3f-85b6d7ce6b20 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.295220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8e8c9-1050-4199-9b3f-85b6d7ce6b20 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.306058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8e8c9-1050-4199-9b3f-85b6d7ce6b20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f4b8e8c9-1050-4199-9b3f-85b6d7ce6b20 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:45.311236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:45.312759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:45.313601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 190/385] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:45.314256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.314540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.314731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.332148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:45.333356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:45.349960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe5cd5-85cf-4a03-923d-f2fddbf68189 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.383866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.384460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.384738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.385007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.385671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:45.385977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.387338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.398890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 196/387] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:45.399016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 191/387] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:45.399598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.406424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.466630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.506937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.510347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5703b9e-411e-4583-8c3d-b16083f34435 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.514374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d0dec-cfc9-46d3-9824-929739363b46 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.544782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.545343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.545627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.545896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.553295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.553479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.554207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.554633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.554756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.555248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.556086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:45.556792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.558913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:45.566959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:45.567873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 192/388] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:45.568767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.578493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 197/389] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:45.579368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.674911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f2ae3e-271e-4300-8bb4-61bbb4a4aea4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.684068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa92663-641a-4a9f-b62d-2e48bfb4006b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.706945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.707667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.708271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.708573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.708855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.709615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:45.709919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.711299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.717897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.719068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.719347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.719621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.719989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 193/390] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:45.720783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.729825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.731294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.744544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:45.744679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:45.745573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:45.747062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 198/391] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:45.747924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.822710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3bed9d-a6ba-4537-98eb-bcbe0c8c49c8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.846982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2b2748-ac47-44c6-bab6-e8b57f1af19c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:45.856073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.856677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.856960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.857234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.857990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:45.858297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.860370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.880019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:45.880630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:45.880908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:45.881177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:45.887734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:45.889181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:45.896088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 194/392] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:45.896922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:45.901743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:45.902774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 199/393] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:45.903784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:45.949560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:45.982819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.002920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7704eba6-fd0a-4f47-93a7-8075a072efab None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.007693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9793ea-a2e2-47bd-9449-daa91243f10e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.039629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.040251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.040532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.040802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.041580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.042203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.042475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.042752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.050791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.052258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.053092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.054557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:46.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:46.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:46.069108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:46.069108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.069108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:46.069271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:46.070273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:46.071110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:46.071110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.071110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:46.071280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 195/394] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:46.071280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.074054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 200/395] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:46.074742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.167217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.170670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03102f-72bf-4fcd-b985-926f64f10f5a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.174310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac3207-0559-480e-a872-7d1b5b00ec02 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.200550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.205828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.206433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.206714 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.206986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.207674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.207787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:46.208006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.208296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.208577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.208854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.209470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.220540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.222002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.226352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 196/396] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:46.227160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:46.237097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:46.238105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:46.239378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:46.239378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.239378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:46.239378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 201/397] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:46.239378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.329632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e040b0cb-2d38-4560-90ac-95ade32f7e26 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.341559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0a380-1089-46c7-ab3e-38a0f946abd6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.361951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.362538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.362817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.363088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.363771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:46.364073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.365543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.376102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.376702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.376981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.377258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.377925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:46.378226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.379578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.384364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 197/398] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:46.385259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.396977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 202/399] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:46.396977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.403572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.435952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.485573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e60f98-e869-446f-ad9b-47b50d5b6ea4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.491503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c02377-bb29-4837-9680-edcd548de246 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.518404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.518996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.519273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.519541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.520174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:46.520476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.521876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.525127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.525771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.526051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.526321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.533151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 198/400] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:46.533977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.541371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.542820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:46.557788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:46.558962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:46.560022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:46.560022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:46.560022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:46.560022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 203/401] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:46.560022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.622706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.639827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d591b70-956c-436c-8548-f60a8ed6fd2d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.650002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.664250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eecf7b-e487-4426-8264-a6c6677b8562 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.674187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.674794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.675073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.675348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.675987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:46.676291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.677716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.689065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 199/402] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:46.689877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.697450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.698095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.698416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.698632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/5e2c96c17470485f90e95e3e5bb87ffa/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.712470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=5e2c96c17470485f90e95e3e5bb87ffa, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.713941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.726447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 204/403] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:46 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/5e2c96c17470485f90e95e3e5bb87ffa/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:46.727235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.817877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ded58-8bfc-4f8a-9cba-0d548bdb41f0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.848106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eac345d-c64a-46ba-85e9-6957758d9331 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:46.858913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.859536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.859815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.860087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.860746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.862270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.863196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.885932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 200/404] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:46 2022] POST /identity/v3/users => generated 247 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:46.886841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:46.914817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:46.915497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:46.915832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:46.916109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:46.924481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:46.926113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:46.932023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:46.941915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:46.943081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:46.944063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 205/405] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:46.949026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.043646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b796f2c1-0a27-4358-9eb5-e135ef7905a9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.065557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4704d4e-187f-44d1-be51-fe593ab9608c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.086647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.087117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.087260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.087456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.088114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:47.088420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.089870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.103926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 201/406] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:47.104587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.109719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.110298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.110564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.110833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.119390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.122171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:47.138180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:47.139216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:47.140259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:47.140883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 206/407] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:47.141178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.177205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:47.227095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:47.250107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac07ac6-74c7-417b-a674-18a9291d49f8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.262716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bbaa5e-e4ab-4ff2-843c-8a2891db8368 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.297151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.297932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.298432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.298907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.299836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:47.300244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.302016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.307697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.308328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.308606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.308881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.318053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.319881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.321496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 202/408] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:47.322665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:47.332504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:47.333628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 207/409] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:47.334784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.440149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce6accc-bf48-4910-aff4-a9905f5f5a5a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.444006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:47.459564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83599ae1-f861-4f7b-b6bf-45e319605d04 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.485183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.486696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.487157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.487600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.488634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:47.489427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.491494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.497008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.497730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.498113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.498452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.507758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 203/410] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:47.509270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.510460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.512057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.513494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:47.530793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:47.531809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:47.532653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:47.532653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:47.532653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:47.532653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 208/411] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:47.532653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.625402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997a575-506a-4389-96d1-c4c89a555e02 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.657702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c1c7d5-090b-4daa-8834-189b11930e53 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.661264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.662260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.662743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.663307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.664255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:47.664769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.666531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.681062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 204/412] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:47.682323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.699282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.699859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.700203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.700463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/1fb121d7e44f4f868bc28dda9647bbee/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.715577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:47.720215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=1fb121d7e44f4f868bc28dda9647bbee, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.721706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.740072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 209/413] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:47 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/1fb121d7e44f4f868bc28dda9647bbee/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:47.740876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.756413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:47.801958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31d465-beac-4458-a2e9-2c27ab25c9c6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.846499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.847163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.847464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.848110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.848760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:47.849060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.850525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.868847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 205/414] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:47.869642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad65fb28-9fae-456a-9f43-ef2e500813c5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:47.876172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:47.915805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:47.916828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:47.917346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:47.917816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/fde1aa258e9b400983bc8cffa1a8b99c/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:47.937588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=fde1aa258e9b400983bc8cffa1a8b99c, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:47.939932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:47.963587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 210/415] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:47 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/fde1aa258e9b400983bc8cffa1a8b99c/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:47.990507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9bcf-6d15-4c25-90fb-2372742b22b7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:48.021767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:48.035542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:48.036198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:48.036473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:48.036740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:48.044828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:48.048362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:48.062391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:48.063462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:48.064323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 206/416] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:48.069643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:48.126070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:48.211365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcc43d-180c-48af-869e-ba302e5a8e15 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:48.250137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:48.250968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:48.251253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:48.251530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:48.252228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:48.252534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:48.254009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e369b4e1-564d-4dab-b298-dbfbf5dc530e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:48.276217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 211/417] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:48.282495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:48.439401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086295-4113-4811-9096-953b00cdf80e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:48.485272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e564df-c524-4b21-9fd0-f8e4b6991fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:48.486626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e564df-c524-4b21-9fd0-f8e4b6991fdc None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:48.487042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e564df-c524-4b21-9fd0-f8e4b6991fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:48.487450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e564df-c524-4b21-9fd0-f8e4b6991fdc None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/afdf63d4c79649a48cddc9f64e10e3e3/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:48.508533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e564df-c524-4b21-9fd0-f8e4b6991fdc None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=afdf63d4c79649a48cddc9f64e10e3e3, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:48.510206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e564df-c524-4b21-9fd0-f8e4b6991fdc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:48.540320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 207/418] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:48 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/afdf63d4c79649a48cddc9f64e10e3e3/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:49.297035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1de21-7824-49e5-81f3-04fbdebe43c2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:49.298073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1de21-7824-49e5-81f3-04fbdebe43c2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:49.298557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e1de21-7824-49e5-81f3-04fbdebe43c2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:49.300383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 212/419] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:49.305903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665d34-f3f2-461c-bb17-6dafe4882fa8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:49.306417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665d34-f3f2-461c-bb17-6dafe4882fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:49.306889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b665d34-f3f2-461c-bb17-6dafe4882fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:49.330652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7b665d34-f3f2-461c-bb17-6dafe4882fa8 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:49.331866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b665d34-f3f2-461c-bb17-6dafe4882fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:49.377143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:49.439128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 208/420] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:49.499734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d510ab2-0e89-4197-9785-5a5c8fc478ec None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:49.500095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d510ab2-0e89-4197-9785-5a5c8fc478ec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:49.500314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d510ab2-0e89-4197-9785-5a5c8fc478ec None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:49.501932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 213/421] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:49.507228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:49.507682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:49.507777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:49.538379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:49.539503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:49.630940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 209/422] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:49.738852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d510ab2-0e89-4197-9785-5a5c8fc478ec None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:49.867002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d510ab2-0e89-4197-9785-5a5c8fc478ec None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:49.868391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:49.903064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:49.903678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:49.903952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:49.904220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:49.915353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:49.920601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:49.921615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:49.922430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 214/423] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:49.928039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.023199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:50.031100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aaa96a-8dcb-43be-9482-b55804a2f0de None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.069981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.070584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.070857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.071124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.071781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:50.072077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:50.073474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:50.083198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 210/424] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:50.088759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.145097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.256336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1605f3-bfcd-4410-a07b-1be429d8265c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.323150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e65ba9-fd95-4068-8c7c-d99e8e240d33 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.324054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.324725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.325002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.325268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.345432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:50.347306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:50.375808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:50.376771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:50.377652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:50.377652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:50.377652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:50.378404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 215/425] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:50.384916 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.403394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.403953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.404231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.404501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.417677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:50.428357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:50.437498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:50.438411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:50.439649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 211/426] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:50.447504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.479813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:50.539158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be195a1d-af93-4612-822a-8ea9cd4460f8 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.574376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.574976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.575252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.575523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.576156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:50.576450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:50.578457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070932e-c8ab-4815-8ebf-da50f4214174 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:50.586080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2772555c-0f15-4b61-b4d6-6a1ac93862ac None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.590718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 216/427] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:50.591936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67c90b-7b7c-4c9a-a9eb-9e7cc4854b9b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.592219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67c90b-7b7c-4c9a-a9eb-9e7cc4854b9b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.592493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc67c90b-7b7c-4c9a-a9eb-9e7cc4854b9b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.597442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 217/428] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:50.598208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67c90b-7b7c-4c9a-a9eb-9e7cc4854b9b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.636030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.636853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.637131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.637427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.638185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:50.638495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:50.640942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857efd9-9ebc-49e6-b20f-d26e6ba78c99 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:50.656196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 212/429] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:50.656196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.656196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.656196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.674049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:50.675267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:50.708551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:50.747158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc67c90b-7b7c-4c9a-a9eb-9e7cc4854b9b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.753721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 213/430] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:50.754666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.772854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:50.792327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.793823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.794398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.794841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.804509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:50.806763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cfaddacd-4da0-475b-8b11-0c00d7406846 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:50.823666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:50.824751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:18:50.825799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:50.831844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 218/431] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:50.831844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa975-c452-4785-a006-5994460559dc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.831844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa975-c452-4785-a006-5994460559dc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.831844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aa975-c452-4785-a006-5994460559dc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.833083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 219/432] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:50.833767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa975-c452-4785-a006-5994460559dc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:50.891625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202573f-5e08-4f9e-b66b-6b2c5f34d715 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.934974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.935531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.935809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.936085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.948818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:50.950407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:50.954223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aa975-c452-4785-a006-5994460559dc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-621465c3-4a75-4424-a895-8164cbf0bb0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:50.966562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:50.967487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:50.968395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:50.968395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:50.968395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:50.972547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 214/433] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:50.973652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.973949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:50.974219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:50.992311 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:50.993536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:50.997645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:50.998505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:50.998974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.000095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.001207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:51.001727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.003955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.007049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.032855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 220/434] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Oct 12 12:18:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:51.033349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.057853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.073430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 215/435] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:51.074623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.142938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82da4efe-499c-4dba-806f-224517d383e6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.181914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.182478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.182758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.183024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.183756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:51.183973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.185520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.204922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 221/436] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:51.205930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.212455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bf2bea-79cc-446e-8ebc-eda42f89494c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.258116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.259104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.259573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.260050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.272689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:51.273040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:51.279203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:51.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 216/437] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:51.281189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.329014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226a8333-3e98-49fe-9bf8-ee8f35d12e8a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.335989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.367159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461857c0-5ed8-4281-bd43-a6ca322756ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.367765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461857c0-5ed8-4281-bd43-a6ca322756ce None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.368046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461857c0-5ed8-4281-bd43-a6ca322756ce None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.368318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461857c0-5ed8-4281-bd43-a6ca322756ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.387995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a23a2-da7e-40d5-b8c2-af97aca1aef3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.420574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.421702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.421975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.422251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.429156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.430651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.434668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461857c0-5ed8-4281-bd43-a6ca322756ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.436162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461857c0-5ed8-4281-bd43-a6ca322756ce None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:51.442514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:51.443481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 217/438] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:51.444356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.491272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.539189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 222/439] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:51.540446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e8eb7-dc3f-442a-bc0b-2e27c19bd0b6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.540731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e8eb7-dc3f-442a-bc0b-2e27c19bd0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.541019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e8eb7-dc3f-442a-bc0b-2e27c19bd0b6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.542491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 223/440] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:51.543094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e8eb7-dc3f-442a-bc0b-2e27c19bd0b6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.543968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81ddc8e-2e76-4898-8d7f-110fb260975d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.555426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.578896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.579497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.579774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.580048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.580702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:51.580999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.582412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.590682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 218/441] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:51.591457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.644308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e8eb7-dc3f-442a-bc0b-2e27c19bd0b6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.677822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0939fb-3014-44f5-afb4-da61099b8034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.678437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0939fb-3014-44f5-afb4-da61099b8034 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.678716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0939fb-3014-44f5-afb4-da61099b8034 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.678987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0939fb-3014-44f5-afb4-da61099b8034 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.689373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0939fb-3014-44f5-afb4-da61099b8034 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:51.691491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92634c-125d-475c-9655-f506bc8fc572 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ce0939fb-3014-44f5-afb4-da61099b8034 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:51.694127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:51.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:51.696074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 224/442] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:51.697107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.697107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.697107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.714111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:51.715323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:51.723086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.727477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.728150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.728445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.728712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.729464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:51.729768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.731187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.749059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 219/443] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:51.749984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.778668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.786662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 225/444] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:51.787566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.856301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d44fb3-76c2-4ebf-82b8-a4bda1403e9c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.891958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.892759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.893038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.893328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.895444 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293413fb-6fe7-42aa-8667-b17ec60892f9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:51.903253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.904710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:51.920926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:51.922624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:51.923506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:51.923506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:51.923506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:51.923506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 220/445] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:51.923718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:51.938480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:51.939161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:51.939501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:51.939723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:51.950622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:51.952083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:51.954119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:51.969362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:51.970269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:51.971141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:51.971141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:51.971141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:51.971141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 226/446] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:51.971770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.017799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.038599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb091-e2bd-4b93-8d93-5a0f75fbba30 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.073464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.074060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.074339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.074608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.075264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.075564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.076920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.079048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73a2a9e-8620-4ea3-a0ae-24d5d37277c9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.084797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 221/447] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:52.085922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.111782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.112397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.112674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.112946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.123605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.125909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:52.139969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:52.140910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:52.141835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:52.141835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:52.141835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:52.141835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 227/448] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:52.146095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.186017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1df29b-02d6-4f73-aa7b-8ee1bd4a36ee None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.193622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.220602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.221202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.221518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.221795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.222431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.222732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.224107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.236027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 222/449] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:52.236820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.240105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.248697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36023f3-3289-4dd0-a91c-250f764dcf02 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.289169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.289806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.290086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.290361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.291013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.291307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.292725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.304702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 228/450] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:52.305516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.339138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990f8c4-3357-4e61-a92a-2d86010e281a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.372195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.373092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.373401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.373672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.384000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:52.388733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:52.390006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:52.390896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 223/451] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:52.391562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.405059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b20678-726d-40d8-b4fa-c5f45fbde9a5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.412802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.440132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.440762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.441043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.441337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.451693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.453123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.453785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:52.470682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:52.471728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:52.472606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:52.472606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:52.472606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:52.472606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 229/452] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:52.472606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.494016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28da54ca-7283-4e45-9e8d-fd3a37033b69 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.529803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.531302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.531302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.531636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.532191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.532492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.533914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.544723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 224/453] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:52.545532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.571798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e62611-6893-4163-9b69-20bc0b3f7e62 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.604489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.605108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.605410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.605686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.612875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.614337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:52.625834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:52.626743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:52.627582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 230/454] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:52.628045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.643801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a9bb2-6ac0-4d6d-b0e8-5cbb04f74388 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.646716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.668076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.680866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.681533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.681813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.682082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.690434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.691889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:52.702994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:52.704077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 225/455] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:52.704956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.731514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef604485-0f49-4f4f-b3ed-e8a3ca0f5d56 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.766020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.766620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.766902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.767199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.767854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.768154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.769542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.777858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 231/456] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:52.778997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.805597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2f765-8ec2-4b6c-86fb-5f6dc98b7124 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.839013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.839606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.839878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.840152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.840775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.841069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.842481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.854288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 226/457] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:52.855104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.859353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.879021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3f3310-8591-4147-b1eb-d6e4cf853ee4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.899677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:52.914376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.915004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.915287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.915564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.922111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.923554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:52.934580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:52.935471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:52.936328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 232/458] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:52.936948 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:52.956762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84bd98c-96f9-44c7-a6cf-535ca9390d9a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:52.990676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:52.991273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:52.991552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:52.991823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:52.992527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:52.992827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:52.994283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.005569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 227/459] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:53.006346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.038963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a85993-b8c9-444b-8408-8821cf847d14 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.070313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc099998-581c-45eb-8166-110096c97353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.070901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc099998-581c-45eb-8166-110096c97353 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.071175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc099998-581c-45eb-8166-110096c97353 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.071443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc099998-581c-45eb-8166-110096c97353 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.072158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc099998-581c-45eb-8166-110096c97353 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:53.072456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc099998-581c-45eb-8166-110096c97353 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.073919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc099998-581c-45eb-8166-110096c97353 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.083358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.096628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 233/460] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Oct 12 12:18:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:53.096628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc099998-581c-45eb-8166-110096c97353 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.104159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447d3a-23bf-48fd-b207-2a3a9327697f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.106449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.137502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.138168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.138451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.138715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.149611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.151057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:53.165829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:53.166739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:53.167586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:53.167586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:53.167586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:53.167586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 228/461] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:53.167800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.192644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc099998-581c-45eb-8166-110096c97353 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.226057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.226663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.226935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.227198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.234125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.235589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:53.247027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:53.248000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 234/462] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:53.248906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.274177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332de3d-30a5-4f6d-b58b-4e20dbebf3a6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.308124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.308731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.309010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.309351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.310024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:53.310136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.310332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.311903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.323817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 229/463] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:53.324615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.341658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.354490 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afafbfeb-d1af-4db4-a022-afe239642cc9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.388862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.389504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.389840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.390120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.390834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=5e2c96c17470485f90e95e3e5bb87ffa) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:53.391143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.392590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.398296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 235/464] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/role_assignments?user.id=5e2c96c17470485f90e95e3e5bb87ffa&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:53.399077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.429400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d76c7-2d35-479a-a9ac-5c7ff11fc49f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.468690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.469378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.469660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.469933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.521209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2054a30b-44ef-470f-87fe-d0894d8b61d1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.548120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.555017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.556514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.561381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.562299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.562776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.563263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.564115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:53.564621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.566235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.576321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.582502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 236/465] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:53.583363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.670689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 230/466] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:53.671557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.697261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730403e1-a67f-428b-91d5-52f3d8d2a797 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.734240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.734835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.735115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.735584 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.736109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:53.736411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.737899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.759881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 237/467] 10.222.0.17 () {56 vars in 1137 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:53.759881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.788580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd513f2-589a-4aba-ba56-9474adb806fa None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.799970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.806829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:53.826509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.827668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.828024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.828277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/e7f71a2997894b309bd2333a972107d1/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.843507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=e7f71a2997894b309bd2333a972107d1, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.844963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.858260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 231/468] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:53 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/e7f71a2997894b309bd2333a972107d1/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:53.859540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:53.879813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488f1bf-64c4-44a0-99cd-aa14402b1450 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:53.927821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:53.929384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:53.929922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:53.930393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:53.940024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:53.942064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:53.956186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:53.957130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 12 12:18:53.958069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:53.960680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 238/469] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:53.961845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.011558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6552112-0b58-44be-b9ee-6f51e0a2ae2f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.052557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.053532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.054006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.054467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.066618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.068397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.084054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.087561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57030d-5265-43ca-8b39-1576987e9f84 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:54.088658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:54.089828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:54.090655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:54.090655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:54.090655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:54.094096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 232/470] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:54.095140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.125763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.137983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.142106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.142496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.142876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.216262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1a52a-2ce4-49da-8696-e30575cb6dad None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.232406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.234287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.277866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.279080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.279499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.280917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.282113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:54.282555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.284901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.309188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 233/471] 10.222.0.17 () {56 vars in 1139 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:54.311353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.346721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.351048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 239/472] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:54.356389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.413762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.431041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a04135-af83-494e-a09e-ddf1615e337e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.461803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7a985-7524-4f8d-8be1-aa28d0d8b4ca None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.476821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.477991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.478445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.479100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.480279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:54.480776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.482847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.498528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.498997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 234/473] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:54.499768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.500126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.500401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.501246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.574096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.576387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.578341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.641289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b53d-c969-4d27-a4a2-584c840ca149 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.683332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.683956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.684370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.684704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.688575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.692273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 240/474] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:54.693296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.703190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.705932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:54.726254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:54.727218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:54.728060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:54.728060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:54.728060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:54.730408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 235/475] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:54.731335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.820118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:54.824690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338f79c-59ce-4cec-9289-78587db2e02b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.853985 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cade9fd-bf51-4e49-8bb4-1a4b6f08ab65 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:54.871680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.872845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.873292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.873736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.883391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.885918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.889746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:54.890550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:54.890960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:54.891326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:54.898535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:54.899541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:54.900583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:54.901947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 241/476] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:54.902790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.912157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:54.913785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:54.929450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:54.930545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:54.931580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:54.931580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:54.931580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:54.932889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 236/477] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:54.938046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:54.981292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.030884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aeb25c-b808-4864-97c7-3070ed1255f3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.049621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b11f42e-b29b-48aa-b81d-5ecbe7e870fd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.076472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.078662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.079084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.079495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.080542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.081434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.083248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.086103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.086748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.087124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.087494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.088319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.088768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.090408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.098099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 242/478] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:55.099120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.109443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 237/479] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:55.110536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.127654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.212407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4a1586-248a-4c03-86db-74343fc9e693 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.228101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.231825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36faa012-fabb-4055-a36e-cc1a613e912a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.250411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce993-d569-432f-b37e-123174631351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.251186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce993-d569-432f-b37e-123174631351 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.251538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce993-d569-432f-b37e-123174631351 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.251880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bce993-d569-432f-b37e-123174631351 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.252738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce993-d569-432f-b37e-123174631351 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.253106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce993-d569-432f-b37e-123174631351 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.254635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bce993-d569-432f-b37e-123174631351 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.273059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 243/480] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:55.274019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce993-d569-432f-b37e-123174631351 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.275947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.276774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.277224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.277711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.284904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.286612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:55.298758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:55.299598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:55.300439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:55.302745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 238/481] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:55.303708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.367271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.390140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bce993-d569-432f-b37e-123174631351 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.405121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fdcb5d-a52a-4a3c-a6ac-661d7cc7de79 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.427815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.428421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.428700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.429051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.429904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=1fb121d7e44f4f868bc28dda9647bbee) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.430435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.431733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.438347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 244/482] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/role_assignments?user.id=1fb121d7e44f4f868bc28dda9647bbee&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:55.439026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.442079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.442933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.443359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.443769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.451746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.453736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:55.465951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:55.467013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:55.468519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:55.470567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 239/483] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:55.476913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.503695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.564234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d86a9-cc2f-45a7-8a05-1e0df9b2ce4f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.596439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbf1c3-2a18-4b8b-a1e3-a6cf850bfd9b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.602696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.603315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.603594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.603867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.604572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.604877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.606504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.621880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 245/484] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:55.628418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.632471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.638304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.639285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.640402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.641038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.641961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.642509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.644128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053f4642-3350-4b37-93af-160bffeb738b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.658367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 240/485] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:55.659874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f068a-64a9-4fa3-a90e-a62c3388aae2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.660321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f068a-64a9-4fa3-a90e-a62c3388aae2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.660730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f068a-64a9-4fa3-a90e-a62c3388aae2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.662418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 241/486] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:55.667589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.668131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.668548 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.693216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:55.695907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:55.762426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad5dc4b-b51d-476e-8ca3-5ef8011b9794 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.793335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 242/487] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:55.793335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:55.802810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:55.808609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.809894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.810181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.810652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.811444 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=fde1aa258e9b400983bc8cffa1a8b99c) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.812301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.813946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.822581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 246/488] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/role_assignments?user.id=fde1aa258e9b400983bc8cffa1a8b99c&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:18:55.910387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010af3e-a9f7-4111-b8b6-d7a58e596ae0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:55.950939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:55.951559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:55.951842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:55.952632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:55.954034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=afdf63d4c79649a48cddc9f64e10e3e3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:55.954350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:55.955967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:55.963194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 243/489] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:18:55 2022] GET /identity/v3/role_assignments?user.id=afdf63d4c79649a48cddc9f64e10e3e3&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:56.094048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:56.175486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:56.218443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158af14-52a6-4586-9620-de91742c8172 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:56.255206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c46a9d-0a7b-4b64-bce7-87c0c9a9e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:56.255838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c46a9d-0a7b-4b64-bce7-87c0c9a9e490 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:56.256117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c46a9d-0a7b-4b64-bce7-87c0c9a9e490 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:56.256389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c46a9d-0a7b-4b64-bce7-87c0c9a9e490 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:56.268850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c46a9d-0a7b-4b64-bce7-87c0c9a9e490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d5c46a9d-0a7b-4b64-bce7-87c0c9a9e490 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:56.274233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:56.275142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:56.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 247/490] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:56.284220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:56.295675 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538283ae-f4cb-49b7-8be3-8954abc049a4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:56.296560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538283ae-f4cb-49b7-8be3-8954abc049a4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:56.296839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538283ae-f4cb-49b7-8be3-8954abc049a4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:56.299540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 248/491] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:18:56.309352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:56.309352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:56.309352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:56.329814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:56.331003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:56.398268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:56.437160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37020f68-342e-4c43-962e-c02bb92ebe07 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:56.457460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 249/492] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 12 12:18:56.495227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:56.496445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:56.496923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:56.497443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:56.498315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:56.498811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:56.500405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:56.513912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 244/493] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:18:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:56.521088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:56.586995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:56.631211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d9eba1-7ef0-4e7e-8c29-08c30feda611 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:56.670015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:56.670648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:56.671437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:56.671703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:56.688839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:56.690301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:56.705901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:56.707242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:56.708400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:56.708400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:56.708400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:56.708400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 250/494] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:56.719077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:56.846691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa1759-87c4-4649-94d1-5c4eb78c9a6a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:56.889473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:56.890611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:56.890891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:56.891180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:56.891799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:56.892189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:56.893586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:56.896091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:56.911362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 245/495] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:56.916942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:56.917547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.034573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be38b28f-1d81-4d89-a8cd-dd04d2166e27 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.048874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:57.077051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.077807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.078080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.078345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.086402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:57.087846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:57.100243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:57.101152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:57.102264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 251/496] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Oct 12 12:18:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:57.109781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.129837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5422b263-8043-4e93-b1be-d62cf4f37fbc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.184769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.185699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.186125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.186615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.197442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:18:57.202582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:57.203826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:18:57.204854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:57.207195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 246/497] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:18:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:57.214243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.286160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:57.323770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890f987-1664-4b1b-a30f-21a8928e92ff None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.334061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684ae26f-6da7-4fcb-ac62-6bd15c35b860 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.376885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.377849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.378699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.379193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.380203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:57.380735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:57.382651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:57.402263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 252/498] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:57.408646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.415685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.416569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.419273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.419729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.420650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:57.421289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:57.422870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:57.444434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 247/499] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:57.458759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:57.464425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.551370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c43327-2c9a-4b3f-b61e-2ed75d8b0a27 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.594308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.595403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.595934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.596423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.608035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:57.610035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:57.627174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:57.628197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:57.629613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:57.629613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:57.629613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:57.632335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 253/500] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:57.640994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.681163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e85997-3fa4-4be0-ae50-0cb390070191 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.717474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:57.750189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.750802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.751083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.751351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.775507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:57.776963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:57.798532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:57.808170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:57.809268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:57.810738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:57.810738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:57.810738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:57.810738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 248/501] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:57.816387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:57.836835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e253fa-0817-4ad4-964c-40e9a1fcec9a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:57.879371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:57.880020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:57.880305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:57.880579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:57.881216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:57.881586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:57.883054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:57.896375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 254/502] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:57.901347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.008686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17f529a-b66c-4a8a-94ae-e7361be1d261 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.073337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.073886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.074164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.074436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.075845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:58.076155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.078476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.081460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c35cb7-ef41-4dac-83a7-cf30f297eb63 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.104992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 249/503] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:58.111086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.148103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.148792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.149070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.149357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.162773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.163355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:58.164229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.171471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:58.191283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:58.192262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:58.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:58.194055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 255/504] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:58.199127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.272953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c64c-5fbb-435e-9e77-9bf19165b144 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.321997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.322641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.322921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.323194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.333089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.334510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2c032f2c-d0d7-4d1d-a21e-e0db7d19e3d3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:18:58.350299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:58.351282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:18:58.352195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:58.353456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 250/505] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:18:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:18:58.354744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690682b-7b3b-4c64-b7cf-ee7c62caa935 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.355026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690682b-7b3b-4c64-b7cf-ee7c62caa935 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.355299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690682b-7b3b-4c64-b7cf-ee7c62caa935 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.356745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 251/506] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:58.359598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690682b-7b3b-4c64-b7cf-ee7c62caa935 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.374961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665ea18f-752c-4462-b9b6-abb0e60abc42 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.415855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.417131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.417443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.417718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.418888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:58.419208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.422106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d38e6-8629-4949-a047-a94cddbff926 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.438383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 256/507] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:58.439699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.439981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.440251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.460932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:58.462188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:58.489756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:58.501667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690682b-7b3b-4c64-b7cf-ee7c62caa935 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.534802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:58.546926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.547600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.547878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.548211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.548551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 257/508] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:58.548953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:58.549256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.549397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.550735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ed366-92f3-459d-bca7-86f9ce2a1837 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.573682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 252/509] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Oct 12 12:18:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:58.575412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499be85-6871-470a-8cae-e5fb5db85b6b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.575902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499be85-6871-470a-8cae-e5fb5db85b6b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.576502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499be85-6871-470a-8cae-e5fb5db85b6b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.582664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 253/510] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:58.583847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499be85-6871-470a-8cae-e5fb5db85b6b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.669030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c5ddb-3065-4c1f-9579-5de0f8916aec None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.709620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a202d-e43c-42e2-b505-a931cdf9cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.711009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a202d-e43c-42e2-b505-a931cdf9cee0 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.711725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a202d-e43c-42e2-b505-a931cdf9cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.712184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a202d-e43c-42e2-b505-a931cdf9cee0 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/5e2c96c17470485f90e95e3e5bb87ffa/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.714358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a202d-e43c-42e2-b505-a931cdf9cee0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=5e2c96c17470485f90e95e3e5bb87ffa, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.716146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a202d-e43c-42e2-b505-a931cdf9cee0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.725133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499be85-6871-470a-8cae-e5fb5db85b6b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:58.734495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 258/511] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:18:58 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/5e2c96c17470485f90e95e3e5bb87ffa/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:18:58.736558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57501b1f-87f8-4f5e-b546-43c543882b2c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.737090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57501b1f-87f8-4f5e-b546-43c543882b2c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.737982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57501b1f-87f8-4f5e-b546-43c543882b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.761350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-57501b1f-87f8-4f5e-b546-43c543882b2c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:58.762908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-57501b1f-87f8-4f5e-b546-43c543882b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:58.768171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95feffc-67cc-4d70-94d6-e6f7eaa7d773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:58.768709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95feffc-67cc-4d70-94d6-e6f7eaa7d773 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.768825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95feffc-67cc-4d70-94d6-e6f7eaa7d773 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.769153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95feffc-67cc-4d70-94d6-e6f7eaa7d773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.792095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:58.828593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:58.855974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 259/512] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:58.857485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff89a-5233-4eae-afb1-31fa8c2e2ddf None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.857829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff89a-5233-4eae-afb1-31fa8c2e2ddf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.859196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3ff89a-5233-4eae-afb1-31fa8c2e2ddf None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.860707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 260/513] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:18:58.861554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff89a-5233-4eae-afb1-31fa8c2e2ddf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:58.861876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95feffc-67cc-4d70-94d6-e6f7eaa7d773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:58.863643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95feffc-67cc-4d70-94d6-e6f7eaa7d773 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:58.982104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 254/514] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:18:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:18:58.983487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:58.983780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:58.984056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:58.990578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3ff89a-5233-4eae-afb1-31fa8c2e2ddf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.006601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:18:59.007777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:18:59.032289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.032907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.033184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.033489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.044612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.046335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.056118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:59.064606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:59.065628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:59.066523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:59.066523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:59.066523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:59.066523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 261/515] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:59.066932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.088973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 255/516] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:18:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:18:59.090177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.103032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.179853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4faec-1eae-442a-b296-d01db656575a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.217544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.219336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.219776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.220201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.232860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.234808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:18:59.252502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:59.253690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:18:59.254915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:18:59.254915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:18:59.254915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:59.255126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61454fa2-dfcf-4fa6-85c3-8085227fbd11 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.257648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 262/517] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:59.258612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.306271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.307164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.307606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.308029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.308828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:59.309276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.310843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.323607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 256/518] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:59.324623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.357414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.367362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.372638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ae82a7-503a-4cf4-a087-569fd985cbc2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.410462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.411084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.411357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.411626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.412198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.413607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:59.424743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:59.425851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:18:59.426717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 263/519] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:59.427512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.442758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ee1e4-ebc4-4dea-96e0-557e4e3f3303 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.479007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.479619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.479898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.480170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.480807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:59.481111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.482528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.499411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 257/520] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:59.500288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.536441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba588b95-1ded-42e2-bbd6-d00344574de9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.572582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.573197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.573510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.573783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.574432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.575875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.598205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 264/521] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:18:59 2022] POST /identity/v3/users => generated 254 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:18:59.599135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.611268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.616858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.623796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c5ac45-27bb-4d31-b451-a0fef27d5c9e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.681720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.682634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.683011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.683382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.684405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.686198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:59.700360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:59.701465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:18:59.702388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:59.707291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 258/522] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/services/network => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:59.708390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.736134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7d9f7-296b-4bd7-85df-81f9a42a019f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.779456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.780069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.780393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.780664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.781301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:59.781643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.783064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.788731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 265/523] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:59.789745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.846657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4349b54b-8d68-4c39-91d0-91fa940b7ed7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.882472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:18:59.895092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.896651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.897568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.898192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.908367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.910379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.922283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0606dc4-7c3c-4ed3-a3da-51f4f5f39d5f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:18:59.926915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:18:59.927801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:18:59.928770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:18:59.932084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 259/524] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:18:59.933589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.959349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923977f-7572-4bee-a57d-52241630512c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:18:59.959876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923977f-7572-4bee-a57d-52241630512c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:18:59.960156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923977f-7572-4bee-a57d-52241630512c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:18:59.960430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923977f-7572-4bee-a57d-52241630512c None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:18:59.961055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923977f-7572-4bee-a57d-52241630512c None admin] RBAC: Adding query filter params (name=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:18:59.961395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923977f-7572-4bee-a57d-52241630512c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:18:59.962770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923977f-7572-4bee-a57d-52241630512c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:18:59.968614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 266/525] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:18:59.969365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923977f-7572-4bee-a57d-52241630512c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:18:59.970128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:00.053369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e8009-19fb-4ef9-a26d-24f958dc0e86 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:00.094388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923977f-7572-4bee-a57d-52241630512c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:00.112319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:00.113281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:00.113825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:00.114288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:00.115658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:00.116398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:00.118664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:00.133328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 260/526] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:00.133328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:00.135489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:00.141945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:00.142614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:00.142899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:00.143181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:00.143903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:00.147453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:00.148961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:00.170848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 267/527] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:18:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:00.178545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:00.315303 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:00.344313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf48c4-c791-40b3-938f-b2f628b5a5d8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:00.372615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ca1da-8287-412a-8dde-6bec688bf121 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:00.384377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73791509-584e-4236-8f32-dbc84870829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:00.384987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73791509-584e-4236-8f32-dbc84870829f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:00.385265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73791509-584e-4236-8f32-dbc84870829f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:00.385601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73791509-584e-4236-8f32-dbc84870829f None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:00.386240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73791509-584e-4236-8f32-dbc84870829f None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:00.386541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73791509-584e-4236-8f32-dbc84870829f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:00.387934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73791509-584e-4236-8f32-dbc84870829f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:00.393129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 261/528] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:00.421704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:00.422813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:00.423316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:00.423784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:00.424742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=e7f71a2997894b309bd2333a972107d1) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:00.425251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:00.426963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6269a6ef-9a4a-49a5-96ad-018d29141349 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:00.435504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 268/529] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:00 2022] GET /identity/v3/role_assignments?user.id=e7f71a2997894b309bd2333a972107d1&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:01.041474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3117b2-9fca-44ad-a222-f83a03048385 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:01.041993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3117b2-9fca-44ad-a222-f83a03048385 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:01.041993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3117b2-9fca-44ad-a222-f83a03048385 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:01.043609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 262/530] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:01.049833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:01.050294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:01.050369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:01.077898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:01.079101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:01.141941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:01.164203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 269/531] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:01.555174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3117b2-9fca-44ad-a222-f83a03048385 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:01.690593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3117b2-9fca-44ad-a222-f83a03048385 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:01.738911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:01.739535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:01.739815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:01.740088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:01.755259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:01.760074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:01.760987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:01.762402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 263/532] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:01.767050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:01.904135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf4580-a4fc-4523-91c9-841460bb30ff None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:01.954067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:01.954693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:01.954975 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:01.955251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:01.955926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:01.956230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:01.957642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:01.966820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 270/533] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:01.972204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:01.991281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:02.102444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63592f38-8ed0-49d3-a1b7-611641aeb828 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:02.144650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:02.145700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:02.146176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:02.146662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:02.161374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:02.163110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d86fb9c8-efb8-4fc0-a65b-764816819e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:02.180774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:02.185976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:02.187390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:02.187390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:02.187390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:02.189371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 264/534] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:02.194531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:02.296449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:02.377068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ada4295-549e-4dcb-8c5d-ac160779920e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:02.380318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644db2b3-557a-4d9f-9be6-bb389e081aa8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:02.380993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644db2b3-557a-4d9f-9be6-bb389e081aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:02.381530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644db2b3-557a-4d9f-9be6-bb389e081aa8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:02.384372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 265/535] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:02.388782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:02.389484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:02.390019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:02.410066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:02.411647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:02.433100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:02.433742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:02.434021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:02.434294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:02.434936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:02.435240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:02.436648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:02.458602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 271/536] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:02.463559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:02.477395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:02.497524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 266/537] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:02.644133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8f32-152d-415d-b685-a121a6f14889 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:02.653262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:02.701147 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:02.701715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:02.702074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:02.702484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:02.712422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:02.713909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:02.728849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:02.729812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:02.731333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 272/538] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Oct 12 12:19:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:02.736568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:02.797815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:02.924827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc4fd-a7a7-46e0-b0d2-1120a99e6eb3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:02.980605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:02.981466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:02.981831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:02.982187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:02.983376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:02.983784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:02.985410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baf01fd-f717-454c-a522-5c723d7db8d7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:02.989717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01041680-dcb5-48d7-81e6-e11debcac9d1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.005813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 267/539] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Oct 12 12:19:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:03.007318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4be78-8e5b-4f1e-baaf-88c330b2730e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.007692 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4be78-8e5b-4f1e-baaf-88c330b2730e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.008072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d4be78-8e5b-4f1e-baaf-88c330b2730e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.013484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 268/540] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:03.013484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4be78-8e5b-4f1e-baaf-88c330b2730e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.034862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da2ba2-5b99-428c-a021-e5a3a846b09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.035760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2ba2-5b99-428c-a021-e5a3a846b09c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.036243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2ba2-5b99-428c-a021-e5a3a846b09c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.036706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da2ba2-5b99-428c-a021-e5a3a846b09c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.047647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5da2ba2-5b99-428c-a021-e5a3a846b09c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e5da2ba2-5b99-428c-a021-e5a3a846b09c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:03.053421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:03.054331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:19:03.055157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:03.063065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 273/541] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:19:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:03.064562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.065020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.065500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.079712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.085784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:03.087207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:03.112582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.133912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d4be78-8e5b-4f1e-baaf-88c330b2730e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.163713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 274/542] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:03.164571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.181580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589aaf65-1098-4f9d-9468-a6cfa3f4ccaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.182832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaf65-1098-4f9d-9468-a6cfa3f4ccaa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.183307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaf65-1098-4f9d-9468-a6cfa3f4ccaa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.183831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589aaf65-1098-4f9d-9468-a6cfa3f4ccaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.271558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e53c272-0b8b-43ec-9291-0be2ef436862 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.286225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aaf65-1098-4f9d-9468-a6cfa3f4ccaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.288656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589aaf65-1098-4f9d-9468-a6cfa3f4ccaa None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:03.307656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.308268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.308550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.308826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.309519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:03.309819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.311344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4def2cb-61f5-4a37-914e-9cde95b70945 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:03.325120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 275/543] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:19:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:03.325120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86aec8-3e8f-4547-a47e-61e7d135e355 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.325120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86aec8-3e8f-4547-a47e-61e7d135e355 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.325120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f86aec8-3e8f-4547-a47e-61e7d135e355 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.325120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 276/544] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:03.328648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86aec8-3e8f-4547-a47e-61e7d135e355 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.350843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.355930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.407661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 269/545] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 12 12:19:03.409291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e20dcf-bdbc-48c6-a748-0132327f7809 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.409821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e20dcf-bdbc-48c6-a748-0132327f7809 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.410284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e20dcf-bdbc-48c6-a748-0132327f7809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.428335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-71e20dcf-bdbc-48c6-a748-0132327f7809 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:03.429543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-71e20dcf-bdbc-48c6-a748-0132327f7809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:03.450698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f86aec8-3e8f-4547-a47e-61e7d135e355 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.484700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.485353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.485637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.485912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.496744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.498212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:03.501582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 270/546] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:03.502855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477aa69-1855-408c-a112-92fef70037ff None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.503146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477aa69-1855-408c-a112-92fef70037ff None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.503566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0477aa69-1855-408c-a112-92fef70037ff None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.506348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 271/547] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:03.506348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477aa69-1855-408c-a112-92fef70037ff None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:03.513573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:03.514730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:03.515791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:03.515791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:03.515791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:03.515791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 277/548] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:03.515791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.587988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.610142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.626144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243d6114-d8e7-46e4-bb32-d190b9af881f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.632038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0477aa69-1855-408c-a112-92fef70037ff None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.661981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e371b54-17cc-4449-b4ad-afc083a39d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.662572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371b54-17cc-4449-b4ad-afc083a39d13 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.662847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371b54-17cc-4449-b4ad-afc083a39d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.663113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e371b54-17cc-4449-b4ad-afc083a39d13 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.663763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e371b54-17cc-4449-b4ad-afc083a39d13 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.665121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e371b54-17cc-4449-b4ad-afc083a39d13 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:03.668148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.668774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.669056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.669355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.681548 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.683004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:03.692091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 278/549] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:03 2022] POST /identity/v3/services => generated 240 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:03.692091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.692385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.692978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:03.699295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:03.700320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:03.701238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:03.701238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:03.701238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:03.706358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 272/550] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:03.707330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.714099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:03.715318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:03.809613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 279/551] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:03 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 12 12:19:03.810404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.849467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2e82e3-7547-4938-8432-afaba8b6f118 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.883110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.885041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:03.893239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.893979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.894624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.895235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.895920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:03.896219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.898218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:03.910983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 273/552] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:03.911805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:03.940421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516ffe5-3a5c-4bac-b802-26b07e6908ed None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:03.982920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:03.983800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:03.984225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:03.984674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:03.985640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:03.986202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:03.987636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.005617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 280/553] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:04.009992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.035689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a388794f-ac11-40e3-956e-f3011d70da32 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.080333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.081289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.081682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.082034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.083037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.084664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.104489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 274/554] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:03 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:04.105337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.135362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f27c7-f853-462b-94ca-351af8cc0e68 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.159259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:04.181905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:04.192340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.193654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.194200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.194680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.204746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.206745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:04.221269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:04.222335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:04.223298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:04.226059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 281/555] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:04.227148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.242149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dda55c9-2995-4d55-8f84-346fb8f0d13f None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.292182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.292511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.292786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.293062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.303491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.303491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:04.317720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:04.318702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:04.319630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 275/556] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:04.320112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.359671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd11526-07c4-4561-9afe-187cba62da0c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.398125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.399273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.399810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.400289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.412478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:04.418456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:04.419409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:04.420253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:04.422861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 282/557] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:04.427819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.446585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82dedf-7328-4a05-b133-1eb08abbc14e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.460876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:04.472248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:04.487205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.487823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.488104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.488372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.489106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:04.489455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.490901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.510300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 276/558] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:04.511167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.563983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98f5034-558e-47a7-9d2e-32f8335104eb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.608435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.609600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.610485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.611231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.612281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:04.612791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.614982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.615866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713243e3-18f7-413e-b6f8-88eb8cc25c00 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.632044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 283/559] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:04.633178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.657922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.658562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.658843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.659120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.659784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:04.660142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.661877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.671472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 277/560] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:04.672194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.723261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:04.754049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:04.762808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e7ef4-599a-4d0c-93a6-14a14bb8901f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.805959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.806592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.806874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.807456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.814347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125f5b9-3fa8-4a48-93db-c6b7caf3d1c4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:04.819480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.820953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:04.837353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:04.838394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:04.839300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:04.839300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:04.839300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:04.839300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 284/561] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:04.839955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.859745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:04.861015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:04.861920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:04.862397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:04.863318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=5e2c96c17470485f90e95e3e5bb87ffa) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:04.863846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:04.866311 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:04.876083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 278/562] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/role_assignments?user.id=5e2c96c17470485f90e95e3e5bb87ffa&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:04.877152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:04.995054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.001884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb86129-f44d-4ca4-aee9-dee4e7d74294 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.022682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b43560e-ff70-428a-903c-509ce5731b2d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.049608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.050606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.051831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.052344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.053206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:05.053982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.055756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.071097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 279/563] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:05.079156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.136643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.137177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.137486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.137759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.152830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.154499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.163719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:05.173884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:05.174853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:05.175724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:05.175724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:05.175724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:05.175724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 285/564] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:05.180702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.200888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393621c5-03fc-4aad-bc46-b132ef63e3ce None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.238786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.239728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.240184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.240632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.248226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.250711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:05.265864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:05.266749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:05.267638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:05.270082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 280/565] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:05.277338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.297960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.315051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23833a8d-cdbe-4224-82c8-c9acdf84d779 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.351569 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.352176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.352451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.352719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.353387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:05.353691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.355064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.368596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 286/566] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:05.374840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.391431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fd7b-32d9-4110-85d6-d4b09ac669c6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.427830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.428535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.428814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.429081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.429955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:05.430261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.431786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.445830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.447314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 281/567] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:05.453840 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.494199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78085544-ba52-496e-8e3d-4c14217772de None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.531802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.532439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.532718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.533061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.540478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.541953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.552909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:05.554391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:05.555320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:05.556702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 287/568] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:05.565330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.600132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c0f89-1afc-4603-81c5-608d5b1959e2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.666087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.667001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.667413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.667823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/b840fff8907e4931af6a5133eda97e0d/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.673142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66280cd-a63b-4a3d-b713-c5547dbcc6af None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.691642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.694745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=b840fff8907e4931af6a5133eda97e0d, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.697551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.712794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.713798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.714297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.714810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.715734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:05.716256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.719129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.731965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 282/569] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:05 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/b840fff8907e4931af6a5133eda97e0d/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:05.743435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 288/570] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:05.749700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:05.874541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce264641-dee4-424d-a341-f99ac736759a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:05.917319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:05.928185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:05.928842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:05.929457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:05.929755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:05.947341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:05.948877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:05.968878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:05.970001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:05.970898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:05.970898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:05.970898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:05.974553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 283/571] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:05.975849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:06.105227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eadde86-d9b8-4408-9a21-886b14ff54d3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:06.154131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:06.154745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.155554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.161822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.162514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:06.162822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:06.164220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1de0af2-d3c5-4b8b-a349-1e29434f6fae None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:06.179528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 289/572] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:06.184969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:06.196456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7c3a-3463-42e4-9d30-460079c44d45 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.196741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7c3a-3463-42e4-9d30-460079c44d45 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.196996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b7c3a-3463-42e4-9d30-460079c44d45 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.198477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 290/573] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:06.206254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.206254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.206254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.240642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:06.243119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:06.307906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:06.374547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 291/574] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:06.387370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc1e38c-ea33-4421-ae38-943a58d719a6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:06.459693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:06.460620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.461102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.461596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.469964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:06.471765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a85f64be-4a9a-4cec-9706-d21c2d621cad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:06.489730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:06.490712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:06.491604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:06.497465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 284/575] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:06.500414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:06.533925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39143b1e-e581-4eac-879f-5282715a4798 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.534439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39143b1e-e581-4eac-879f-5282715a4798 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.534959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39143b1e-e581-4eac-879f-5282715a4798 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.536682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 285/576] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:06.541339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.541620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.541895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.572000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:06.575298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:06.633760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:06.682713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c5690e-998b-4a8f-a819-8f8bfa6f18f8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:06.695741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 286/577] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:06.697041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:06.742407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:06.744571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.744996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.745659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.747368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:06.748019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:06.750530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:06.776918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 292/578] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:06.783025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:06.819148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:06.906018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795ecd64-de87-4d6d-87ed-f21be7c2e2a2 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:06.924047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437060a3-a163-42a3-81d0-855758f9b99c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:06.944649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:06.945788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:06.946076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:06.946509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/e7f71a2997894b309bd2333a972107d1/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:06.948874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=e7f71a2997894b309bd2333a972107d1, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:06.950591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:06.967991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 293/579] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:06 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/e7f71a2997894b309bd2333a972107d1/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:19:06.974223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.012691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.013269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.013572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.013844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] PATH_INFO: `/v3/services/network` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.015076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.016538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:07.036600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:07.037783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:19:07.038691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:07.048594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 287/580] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:06 2022] GET /identity/v3/services/network => generated 88 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:07.058055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.096441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:07.116787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d2a794-011c-4391-a083-2999898caf32 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.159695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.160351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.160639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.160922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.161684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.163397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.171660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:07.175995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:07.177027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:07.178729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 294/581] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Oct 12 12:19:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:07.186839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.283770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b48ef-a3bc-45af-b835-d7794cb89656 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.335243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585079fe-069c-42cd-a6fb-d8d3dd7e7120 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.340562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.341175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.341480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.341749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.342379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] RBAC: Adding query filter params (name=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:07.342719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.344098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.355020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 288/582] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:07.362537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.403691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.404295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.404571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.404844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.406353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:07.406662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.409037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.414266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 295/583] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:07.419839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.440126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:07.535810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:07.548175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa444c49-456e-480c-ae17-e991ced1f8aa None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.589637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30099396-e10e-4fc8-b2c3-f7311c7da701 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.602900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.603509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.603788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.604058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.604683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:07.604983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.606393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.614657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 289/584] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:07.620112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.655271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.655879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.656157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.656428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.657049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:07.657376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.659147 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.669755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 296/585] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:07.676393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.784975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2aecb3-1670-4bd9-b12a-9b93100b36dd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.845768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:07.860032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.861098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.861611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.863652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.864934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] RBAC: Adding query filter params (interface=public, service_id=540d10a98cb14b59aee2441a6518e452, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:07.875561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.876751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a840b8-fb2d-4d44-bcd1-15f24dcfcba7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.889754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 290/586] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/endpoints?service_id=540d10a98cb14b59aee2441a6518e452&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:07.891143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda42cd-55a6-4c08-903d-95719e8601a5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.891438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda42cd-55a6-4c08-903d-95719e8601a5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.891720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffda42cd-55a6-4c08-903d-95719e8601a5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.893218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 291/587] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:07.893707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2dcdc4-474f-4212-bc52-6abe73bf16a0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:07.898176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda42cd-55a6-4c08-903d-95719e8601a5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:07.924999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:07.939256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:07.940232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.940739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.941232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.942129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] RBAC: Adding query filter params (interface=public, service_id=9fc012f139d3487985122318a3a78add, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:07.943748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:07.944757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57116694-dcee-4d90-99c4-dae80df11e38 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:07.955140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 297/588] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/endpoints?service_id=9fc012f139d3487985122318a3a78add&interface=public®ion_id=RegionOne => generated 194 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:07.956635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:07.957108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:07.957616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:07.980326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:07.981884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:08.011929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffda42cd-55a6-4c08-903d-95719e8601a5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:08.050455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:08.051159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.051520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.051789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.052571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:08.054092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:08.065542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 292/589] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:07 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:08.067292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:08.069450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 298/590] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:08.218817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:08.237593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbe151-e1e2-42d6-8b9f-2c8539f714ca None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:08.290763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89626f-32f3-4129-a95b-190720ed0257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:08.291442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89626f-32f3-4129-a95b-190720ed0257 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.291728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89626f-32f3-4129-a95b-190720ed0257 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.292007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89626f-32f3-4129-a95b-190720ed0257 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.292737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89626f-32f3-4129-a95b-190720ed0257 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:08.294170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89626f-32f3-4129-a95b-190720ed0257 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:08.301121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 293/591] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:08 2022] GET /identity/v3/services => generated 772 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:08.436009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:08.523065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:08.555883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08383a7-e5dc-45ce-ac4b-6c4218f584d2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:08.580718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa52f96-9b67-4ba2-b95c-763f83dfc6a3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.581231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa52f96-9b67-4ba2-b95c-763f83dfc6a3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.581231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa52f96-9b67-4ba2-b95c-763f83dfc6a3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.582788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 294/592] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:08.589571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.589851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.590129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.596367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:08.597197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.597672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.598081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.598803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:08.600335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:08.610341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:08.611509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:08.612614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:08.615009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 299/593] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Oct 12 12:19:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:08.621566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:08.621790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:08.624974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:08.725642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 295/594] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:08.746997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a02ee5-02b9-4107-9846-d9f0541539af None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:08.783613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:08.784518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.784951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.785405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.786237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:08.786685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:08.788218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:08.790316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:08.794334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 300/595] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:08.800377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:08.917744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:08.944009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae9009b-1f2e-4ed5-bd69-6fc2a5cc1d6c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:08.989993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:08.990877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:08.991301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:08.991713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:08.992570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:08.993042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:08.995031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:09.000656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 296/596] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:09.114041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:09.239463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aacafa-c81d-4abd-97f4-6737a585f3a4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:09.291110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:09.292355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:09.293058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:09.293657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:09.306784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:09.313064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:09.314112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:19:09.314963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:09.317791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 301/597] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:19:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:09.326732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:09.449065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11df20-84b1-4b60-a912-84ec3564a1f9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:09.489754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:09.490764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:09.491274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:09.491806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:09.492888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:09.493442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:09.495024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a953f63-fe03-4f87-992d-a65fe4812707 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:09.505030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 297/598] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:19:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:09.514917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:09.559945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460d853-fc78-4255-964b-30ab8fe4a0c6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:09.559945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460d853-fc78-4255-964b-30ab8fe4a0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:09.559945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460d853-fc78-4255-964b-30ab8fe4a0c6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:09.559945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 298/599] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:09.566587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:09.566876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:09.567156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:09.588506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:09.591883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:09.593614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:09.608928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:09.672743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 299/600] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:09.679408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4862d-bb45-409a-b34f-587a0c380b61 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:09.714157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:09.714707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:09.714989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:09.715320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:09.728002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:09.729501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:09.745123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:09.746380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:09.747455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:09.747455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:09.747455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:09.747455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 302/601] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:09.753738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:09.892481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8406c3d-c7ae-4a64-9017-b39d22344a65 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:09.941858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:09.944780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:09.944780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:09.944780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:09.944780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:09.944780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:09.945357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:09.953146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:09.962895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 300/602] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:09.969862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.040913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.137120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079fc72-9dff-411f-bd2f-b9ee7f960d9b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:10.151577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:10.176838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:10.177773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.178224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.178655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.188648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:10.190419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:10.203596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:10.204740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:10.205734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:10.208417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 303/603] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:19:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:10.216634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.246074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f1ba85-0488-4cc4-b66e-8a90eaec2656 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:10.341741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b612cf0-661a-4183-9c06-a07794ec7d1f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:10.354591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:10.355236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.355519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.355795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.380856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:10.381507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.381784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.382055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.382806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:10.383113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:10.384595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:10.389963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:10.401734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:10.402762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:10.403627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 301/604] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:10.409178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.427114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 304/605] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:10.435669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.495617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:10.531635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:10.616082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e0728-758f-4f78-9d0b-0856cfb4a739 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:10.633051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925450-2bd3-498e-9207-73c983fe64d9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:10.685575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:10.686754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.687069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.687504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.688168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:10.688467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:10.689855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:10.696783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:10.697409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.697686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.697951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.704893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 302/606] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:10.710184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.772895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:10.775242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:10.823228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:10.911659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebbef3-fe52-4622-bfa6-98cde8ca86ec None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:10.925337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 305/607] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 12:19:10.928441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:10.941717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:10.954831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:10.955820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.956484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.956982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.968769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:10.970584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-269e2d3e-7f7d-42be-b3ee-530962f450a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:10.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:10.987745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:10.988652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:10.988652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:10.988652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:10.990956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 303/608] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:10.992771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e480bf-2c6b-4c76-a9cf-af15557fb522 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:10.993340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e480bf-2c6b-4c76-a9cf-af15557fb522 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:10.993858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e480bf-2c6b-4c76-a9cf-af15557fb522 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:10.996044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 304/609] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:10.997072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e480bf-2c6b-4c76-a9cf-af15557fb522 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.081352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e166f8-f26e-4613-ae7a-7c5c6d262874 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.111337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e480bf-2c6b-4c76-a9cf-af15557fb522 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.121163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.129494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.130128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.130407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.130677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.140817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.142311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.150805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.151415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.151693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.151965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.152599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:11.153058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.154461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aa059c-11ca-4775-897c-5e2f09270f39 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c7a4f3d5-771f-4555-87a8-a367f341ab90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:11.159106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:11.160042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 306/610] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.165768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.172913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 305/611] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:11.173450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c26cb-e4b0-4fef-8000-f2d756c9f237 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.173738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c26cb-e4b0-4fef-8000-f2d756c9f237 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.174020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c26cb-e4b0-4fef-8000-f2d756c9f237 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.177366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 306/612] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:11.177366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c26cb-e4b0-4fef-8000-f2d756c9f237 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.185045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:11.186438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:11.219051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.259929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 307/613] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:11.261021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.306673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c26cb-e4b0-4fef-8000-f2d756c9f237 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.356429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.357558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.358072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.359017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.360030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:11.360965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.363085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c976fbc7-c5ce-47e7-85c3-5d3720a9e731 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.379290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 307/614] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:11.380700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.381117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.381552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.384452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef6dcf4-6ae1-4810-a0ab-9caee3138cb5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.399809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.406516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:11.407897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:11.428056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.429328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.429790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.430210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.441164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.443837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:11.459676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:11.460864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 12 12:19:11.461952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:11.464774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 308/615] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:11.465801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.497333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 308/616] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:11.497333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.522378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.615147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220886b-4e61-44d4-8138-c40af2e2466a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.625194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad225e63-1a84-42a6-9add-6c1dcd3352d4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.655998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.656972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.657336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.657683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.658753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:11.659143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.661249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.678257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.679770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.680437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.681279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.685393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 309/617] 10.222.0.17 () {56 vars in 1135 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:11.687091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.696572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.698065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:11.716515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:11.717639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:11.718596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:11.719515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 309/618] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:11.720501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.757049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.780195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.801166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86220fb4-3ea5-45e1-8977-02f437053857 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.830209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f239da-1cf8-4d4c-81ba-cbec1d0d77ee None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.834488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.835115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.835396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.835671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] PATH_INFO: `/v3/services/network` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.836246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.837658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-5e744d5b-ac8d-43e0-8895-c83796ed12bb None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:11.846424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:11.847602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:11.848448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 310/619] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:11.849247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9b650-e33c-4fc2-ac08-cf1e82ae11a3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.849247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9b650-e33c-4fc2-ac08-cf1e82ae11a3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.849680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c9b650-e33c-4fc2-ac08-cf1e82ae11a3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.853353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 311/620] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:11.853353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9b650-e33c-4fc2-ac08-cf1e82ae11a3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:11.867301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.867881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.868164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.868438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.935776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.937534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:11.952674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c9b650-e33c-4fc2-ac08-cf1e82ae11a3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:11.966948 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:11.986640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:11.987237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:11.987515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:11.987786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:11.988461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:11.988764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:11.990220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.001609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 312/621] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.002401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.006209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.040442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 310/622] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:11 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:19:12.041285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.103644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519e57f-c426-4cff-92ca-e0443aa24f22 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.136570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.137122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.137430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.137705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.138272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.138364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bcf469-b76d-401f-90ee-c38a89a5010d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.139688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-4760f41f-42bd-4869-b929-e3eeda0fec4d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:12.148316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:12.149476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:12.150628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 313/623] 10.222.0.17 () {56 vars in 1129 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:12.151638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.151638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.151638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.168963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:12.170182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:12.170294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.175633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.176238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.176514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.176783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.177446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:12.177750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.179113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.183798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 311/624] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.184564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.235703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.241568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 314/625] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:12.242385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.286108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835b78b-37fb-4854-8b87-76f2043841d0 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.326361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929b2a-f867-446a-9474-ac599224203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.327002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929b2a-f867-446a-9474-ac599224203f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.327280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929b2a-f867-446a-9474-ac599224203f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.327714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43929b2a-f867-446a-9474-ac599224203f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.329190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929b2a-f867-446a-9474-ac599224203f None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=b840fff8907e4931af6a5133eda97e0d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:12.329535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929b2a-f867-446a-9474-ac599224203f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.330964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43929b2a-f867-446a-9474-ac599224203f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.336818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 312/626] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/role_assignments?user.id=b840fff8907e4931af6a5133eda97e0d&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.337775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929b2a-f867-446a-9474-ac599224203f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.370480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a28b40b-3b11-4995-a4c4-d09269487682 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.410823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.411896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.412348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.412790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.419637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.427970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.429864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:12.447644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:12.448760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:12.449699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:12.449699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:12.449699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:12.451969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 315/627] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:12.453428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.478291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43929b2a-f867-446a-9474-ac599224203f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.512649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.527401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.529383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.529771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.530354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.531312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:12.531618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.532998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.538445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 313/628] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.539225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.574107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51742bba-85b9-4a8f-8ae3-198ac5573610 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.609709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.610319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.610596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.610863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.611482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] RBAC: Adding query filter params (type=network) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:12.611779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.613132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.622843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 316/629] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.622843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.649638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009d6cdb-e820-49ad-a6c5-2325e0bd1f5c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.674997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.689895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.691535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.692858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.693359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.694477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:12.695247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.697742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.718096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 314/630] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.718096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.730693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210538e6-b280-446e-8684-8a28ced6337b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.745292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:12.768998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.769749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.770097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.770446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.771168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.772610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.787433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 317/631] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:12 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:12.798349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.837723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20999a9-028f-4969-be91-2d8b89b4e4fd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.893125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:12.893999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:12.894373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:12.894811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:12.895606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:12.896010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:12.897603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:12.904363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 315/632] 10.222.0.17 () {56 vars in 1124 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:12.905597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:12.961378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3014920-0165-4cec-bec4-99cd923ba4a7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:12.969688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:13.002198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:13.003082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:13.003521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:13.003798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:13.004465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:13.005901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:13.028185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed57175-a546-424b-823a-82fd98e4cb02 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:13.034993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 318/633] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:12 2022] POST /identity/v3/endpoints => generated 337 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:13.035822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:13.090702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:13.111628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:13.112314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:13.112594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:13.112867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:13.129337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:13.130839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:13.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:13.152738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:13.153683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:13.154789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 316/634] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:13.163439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:13.241786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c5adde-2cd2-4615-9d61-1fa86aa75274 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:13.311988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:13.312895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:13.313502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:13.313920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:13.314832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:13.317558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:13.331993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:13.335738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6533e7a3-594f-40e6-9025-e4c170736a87 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:13.354555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 319/635] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:13 2022] POST /identity/v3/endpoints => generated 346 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:13.402175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:13.402805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:13.403092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:13.403366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:13.404080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:13.404386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:13.406570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321a8eaf-fdde-4b58-9c13-3e783bec3617 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:13.427830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 317/636] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:13.433932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:13.535296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:13.554792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d57bf-eee9-4e10-b05b-8fdc06b4af23 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:13.595149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:13.596130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:13.596476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:13.596809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:13.597647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=e7f71a2997894b309bd2333a972107d1) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:13.598013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:13.599516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed24345f-bbed-4022-ba47-c06fe064e025 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:13.606225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 320/637] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:13 2022] GET /identity/v3/role_assignments?user.id=e7f71a2997894b309bd2333a972107d1&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:14.625696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9ff7-5c42-40ba-ab13-765ca72f1019 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:14.626193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9ff7-5c42-40ba-ab13-765ca72f1019 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:14.626193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521d9ff7-5c42-40ba-ab13-765ca72f1019 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:14.627727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 318/638] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:14.632998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab134250-6b59-4e80-a6e1-56fe0afcd272 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:14.633418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab134250-6b59-4e80-a6e1-56fe0afcd272 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:14.633591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab134250-6b59-4e80-a6e1-56fe0afcd272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:14.652537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ab134250-6b59-4e80-a6e1-56fe0afcd272 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:14.653736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ab134250-6b59-4e80-a6e1-56fe0afcd272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:14.729427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 321/639] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:14 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:14.993413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9ff7-5c42-40ba-ab13-765ca72f1019 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.105663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521d9ff7-5c42-40ba-ab13-765ca72f1019 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.131986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464bb06-10b5-48a6-b5f0-fe03518de3a5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.132327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464bb06-10b5-48a6-b5f0-fe03518de3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.132555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464bb06-10b5-48a6-b5f0-fe03518de3a5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.134144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 322/640] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:15.140386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.140686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.140961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.145891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:15.146515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.146790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.147055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.159725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:15.163020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:15.164189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:15.165581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:15.166517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:15.167355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:15.169966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 319/641] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:15.175718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.196719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:15.248077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 323/642] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:15.286671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f3f4ee-b7bc-418c-b662-b00b587825e6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.322920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:15.323520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.323797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.324069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.324727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:15.325029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:15.326452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:15.335100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 320/643] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:15.340439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.352292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:15.443565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aeee4ac-c64f-4210-918d-dae2ab5c41dd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.477823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:15.478517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.478789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.479059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.490547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:15.492127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:15.508384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:15.509434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:15.510325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:15.510325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:15.510325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:15.510325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 324/644] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:15.515478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.555896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.602654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:15.629701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf3361-ab39-4c42-a48f-a9d8b84eba25 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.631179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:15.669080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:15.669710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.669986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.670398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.671623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:15.671946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:15.673290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae4a64-6d5b-4af1-b65b-e1fbd6debeae None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:15.677811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d5783-48cc-4e22-b955-f240573701c9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.691517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 321/645] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:15.692864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9db25-658a-47b6-8cb7-02c03801ae8b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.693151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9db25-658a-47b6-8cb7-02c03801ae8b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.693457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9db25-658a-47b6-8cb7-02c03801ae8b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.695729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 322/646] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:15.696631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9db25-658a-47b6-8cb7-02c03801ae8b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.721379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:15.722004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.722278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.722542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.723393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:15.724488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d682b1ed-83f1-46cd-8ad0-653b387956d5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:15.733986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:15.734904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:15.735797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 325/647] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:15.737093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.737093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.737256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.756270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:15.757477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:15.819841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9db25-658a-47b6-8cb7-02c03801ae8b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.837359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 326/648] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:15 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:15.837359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.869617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:15.870245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.870519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.871037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.881493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:15.883099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:15.885013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-aa5cc04d-67cd-4555-b0ed-5f2783af2a67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:15.898500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:15.899539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:15.900494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 323/649] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:15.901727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d654d3b-a155-4fb0-bc38-1f2d0e2864a3 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:15.901727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d654d3b-a155-4fb0-bc38-1f2d0e2864a3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:15.901971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d654d3b-a155-4fb0-bc38-1f2d0e2864a3 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:15.903255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 324/650] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:15.907617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d654d3b-a155-4fb0-bc38-1f2d0e2864a3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:15.912051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:15.959640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d955f8-e485-4b88-899b-70f18b86c926 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:15.999680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.000538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.000997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.001446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.002646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.003105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.005062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49e6c8-b1b0-41bf-90d9-36e616718f87 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.011852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 327/651] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.013275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.013896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.014872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.015891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d654d3b-a155-4fb0-bc38-1f2d0e2864a3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.037016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:16.038713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:16.050001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.050598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.050873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.051143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.051857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.052156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.053598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.072117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 325/652] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Oct 12 12:19:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.072840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.126352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 328/653] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:16 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:16.127423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.130308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.179358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.181574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e2562-b9bd-4f0e-86ba-1f30a8a7c8aa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.221679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.222272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.222546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.222811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.223537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.223956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.225339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.229737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580d19-395a-49aa-81d9-6a06e69849a3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.236223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 326/654] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.237083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.268782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.269564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.269965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.270301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.281734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.283184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:16.300173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:16.301335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:16.302737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:16.302737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:16.302737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:16.302737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 329/655] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:16.302737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.348276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3b35d-32e7-470f-8955-4bca50571c24 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.369748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.387196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.387868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.388174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.388442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.389009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.390781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:16.400346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:16.401384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:16.402229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 327/656] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:16.403290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.415608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8c5571-fdbd-488d-9396-b0140ea21144 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.417167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.452366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.453248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.453552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.453817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.454510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] RBAC: Adding query filter params (interface=public, service_id=2e6b01d613d64896922e88bddf333bec, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.454806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.456191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.460978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 330/657] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/endpoints?service_id=2e6b01d613d64896922e88bddf333bec&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.461769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.509923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616b42a-ef19-4002-bfc8-fc2962e5fa94 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.542972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.543566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.543839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.544112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.544733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.545037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.547292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.558542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 328/658] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.559317 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.559733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8831c-ae78-4eea-8536-6911b7bbeb5c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.595810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.596427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.596701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.596967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.597568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.598970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.603678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:16.608013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:16.609716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:16.611015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 331/659] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:16.611812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.643074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.661573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8504-3ca8-47b7-95a6-7eea9a90256e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.694620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.695220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.695498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.695772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.696386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.696689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.698074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.707405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 329/660] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.707405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.710815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c66ba-30ae-4812-8aea-ee93ae4ca1c6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.744610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.745202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.745510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.746040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.746627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.747981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.753457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 332/661] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services => generated 1029 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:19:16.765549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.815808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa46b0-b3e1-4ca7-bb5f-437fb6af0a7b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.827798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.861813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.862820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.863288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.863558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.871309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.872776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.875406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:16.883107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a742db-b495-4fa2-995f-c5df72530ad8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:16.886067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:16.887002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 330/662] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:16.887969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:16.926111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:16.926725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:16.927000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:16.927281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:16.927972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:16.928186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:16.929587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:16.934665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 333/663] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:16.935464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:17.008802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd846e-c434-4be2-9bd5-01b559f56ed1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:17.044888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44679bce-b6fe-40c9-90c2-f1b2af680086 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:17.047598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:17.048328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:17.048611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:17.048928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:17.049590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:17.049897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:17.051315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:17.065340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 331/664] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:17.065340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:17.091530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:17.093049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:17.093387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:17.093668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:17.094491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:17.094801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:17.096430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:17.104886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:17.113090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 334/665] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:17.123405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:17.147647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:17.209048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a1622e-9122-491d-b91b-d8d2fec0f4fd None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:17.244249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34718178-2f23-41ac-9503-19c03cad2768 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:17.255360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:17.256301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:17.256812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:17.257331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:17.258170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:17.258674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:17.260354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3b2014-e8e3-4294-bd2d-3813ff0081c8 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:17.267730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 332/666] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:17 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:17.293926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398793e-5ae7-444b-be4b-4cdefc81d75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:17.294662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398793e-5ae7-444b-be4b-4cdefc81d75c None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:17.294941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398793e-5ae7-444b-be4b-4cdefc81d75c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:17.295213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398793e-5ae7-444b-be4b-4cdefc81d75c None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/b840fff8907e4931af6a5133eda97e0d/roles/a499fa21d23942afae2d8c3066f53b41` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:17.298299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398793e-5ae7-444b-be4b-4cdefc81d75c None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=b840fff8907e4931af6a5133eda97e0d, role_id=a499fa21d23942afae2d8c3066f53b41)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:17.299788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398793e-5ae7-444b-be4b-4cdefc81d75c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:17.318454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 335/667] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:17 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/b840fff8907e4931af6a5133eda97e0d/roles/a499fa21d23942afae2d8c3066f53b41 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:19:18.966762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b6d38-ed92-492c-b9b1-619ca179d829 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:18.968301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b6d38-ed92-492c-b9b1-619ca179d829 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:18.968715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3b6d38-ed92-492c-b9b1-619ca179d829 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:18.970466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 333/668] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:18.976206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb64b-ce95-40f7-ac68-dd2f32f7d83b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:18.977019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb64b-ce95-40f7-ac68-dd2f32f7d83b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:18.977565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cbb64b-ce95-40f7-ac68-dd2f32f7d83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:18.996638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-46cbb64b-ce95-40f7-ac68-dd2f32f7d83b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:18.998198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-46cbb64b-ce95-40f7-ac68-dd2f32f7d83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:19.073410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 336/669] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:18 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:19.295043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06ca4b-46c3-4ca4-912c-b13847d2e81e None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.295955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06ca4b-46c3-4ca4-912c-b13847d2e81e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.296222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06ca4b-46c3-4ca4-912c-b13847d2e81e None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.297842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 334/670] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:19.303218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8cffd-21d4-44bb-a710-db75148be446 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.303510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8cffd-21d4-44bb-a710-db75148be446 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.303789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8cffd-21d4-44bb-a710-db75148be446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.322787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5ca8cffd-21d4-44bb-a710-db75148be446 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:19.323962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ca8cffd-21d4-44bb-a710-db75148be446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:19.325936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:19.351203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06ca4b-46c3-4ca4-912c-b13847d2e81e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:19.409565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 337/671] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:19 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:19.468590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:19.475578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06ca4b-46c3-4ca4-912c-b13847d2e81e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:19.507398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab5311-629a-4744-abde-02c6633473a9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.507707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab5311-629a-4744-abde-02c6633473a9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.508108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab5311-629a-4744-abde-02c6633473a9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.510221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 338/672] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:19.516085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134917f1-f0dc-4279-aed3-d852fb2175bc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.516409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134917f1-f0dc-4279-aed3-d852fb2175bc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.516688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134917f1-f0dc-4279-aed3-d852fb2175bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.518793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:19.519526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.519894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.520274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.520951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:19.522428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:19.532836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:19.533963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 12 12:19:19.534856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:19.536950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-134917f1-f0dc-4279-aed3-d852fb2175bc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:19.538300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-134917f1-f0dc-4279-aed3-d852fb2175bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:19.539492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 335/673] 10.222.0.17 () {56 vars in 1132 bytes} [Wed Oct 12 12:19:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:19.540662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:19.612749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 339/674] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:19 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:19.614274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b496e4-6de4-4de2-b4fd-939737369c15 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.614675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b496e4-6de4-4de2-b4fd-939737369c15 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.615047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b496e4-6de4-4de2-b4fd-939737369c15 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.616599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 340/675] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:19.626724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.627128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.627634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.646534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ffed1-690f-4819-990f-e37a56ee1bbb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:19.649602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:19.655355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:19.684359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e680-a574-4398-8344-32306ab21815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:19.685355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e680-a574-4398-8344-32306ab21815 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.685835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e680-a574-4398-8344-32306ab21815 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.686867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251e680-a574-4398-8344-32306ab21815 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.687932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e680-a574-4398-8344-32306ab21815 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:19.688469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e680-a574-4398-8344-32306ab21815 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:19.690062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251e680-a574-4398-8344-32306ab21815 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:19.695475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 336/676] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:19.700864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e680-a574-4398-8344-32306ab21815 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:19.726876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:19.728861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:19.734382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 341/677] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:19 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:19.735237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:19.832974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251e680-a574-4398-8344-32306ab21815 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:19.842156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a534e508-03e8-4232-b0bd-b26f3a2553db None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:19.878697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:19.879252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:19.879578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.879851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:19.880039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.880145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:19.880391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.880553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:19.881013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:19.881399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:19.881510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:19.882768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:19.883084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:19.889131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 337/678] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:19.895059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:19.906814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 342/679] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:19 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:19.907927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:19.980731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:20.015705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:20.027208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbf7a2-3040-4550-ad1e-2d3ee5f9b211 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:20.044261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cdcd3-6200-4aac-a9fa-ac16405f9ebd None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:20.068060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:20.068685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.068969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.069245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.069957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:20.071928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:20.083457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:20.084368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.084650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.084921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.085650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:20.086979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:20.100138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 338/680] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:19:20.100998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:20.109342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 343/681] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:19 2022] POST /identity/v3/services => generated 241 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:20.249189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57ef25-c42f-4e0f-88da-99bdd31bfabc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:20.297807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:20.298454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.298733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.299004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.302157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:20.312208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:20.318120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:20.319120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:20.319958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:20.322830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 339/682] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:19:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:20.328516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:20.462342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f6b8b-0aa9-48ab-9c31-9268edc16c49 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:20.476691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:20.527441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:20.528063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.528342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.528610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.529276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:20.529603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:20.531048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:20.540000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 344/683] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Oct 12 12:19:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:20.546210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:20.705517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92b325-4fb5-4896-b32d-267352ec7478 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:20.767976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:20.768617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:20.768896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:20.769170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:20.784955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:20.787395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:20.810213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:20.811311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:20.812797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:20.812797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:20.812797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:20.812797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 340/684] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:20.817621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:20.961860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f9baf-9591-4287-896b-b850afe9bc45 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.000479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.002424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.002889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.003307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.004095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:21.004540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:21.006089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:21.020623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 345/685] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:21.026502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.086573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:21.138775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52770e9-b27c-4116-bad2-89ef3e40814c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.175782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.176417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.176697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.176974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.184890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:21.187067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:21.199255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:21.200253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:21.201212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 341/686] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:21.209061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.293730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:21.345248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf10aac3-5db4-4316-bb5d-8dc2fb02dc98 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.387722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.388716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.389810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.390313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.391667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:21.392176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:21.393861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:21.413209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 346/687] 10.222.0.17 () {56 vars in 1149 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:21.421030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.536604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5a6a4-a637-4778-847d-706c475ae354 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.546745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:21.578995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.579595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.579919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.580194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.658669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:21.660133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:21.768013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 342/688] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:19:21.773943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.875934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:21.882461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317656f7-9bc8-4f67-b974-283349f83eee None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:21.918335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:21.918962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:21.919242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:21.919566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:21.930355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:21.931807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:21.946629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:21.947643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:21.949134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:21.949134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:21.949134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:21.949134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 347/689] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:21.953753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:21.963954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:22.057583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114b0d6-cba6-4f99-989d-1544236f36fa None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:22.093298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:22.093981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.094258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.094601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.095168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:22.095464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:22.096840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:22.109084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 343/690] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:22.114644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:22.221625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ad39c6-d323-4e4d-b87b-f4caa1aa7132 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:22.259496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:22.259496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.259496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.259496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.268451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:22.269080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ec4db9c8-dd37-487f-b20c-4fdf237a264d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:22.281849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:22.282773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:22.283655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:22.286141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 348/691] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:22.292010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:22.368030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:22.410977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25344de-267b-4775-baf0-d8d09e451f10 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:22.446293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2103a-c00e-4dc8-9f99-44a3f770ff96 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.446627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2103a-c00e-4dc8-9f99-44a3f770ff96 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.446832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2103a-c00e-4dc8-9f99-44a3f770ff96 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.448008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:22.448396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 349/692] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:22.449090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.449639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.450116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.450999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:22.452148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:22.453973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:22.454100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38da84-6912-40ca-aa5e-0260f7119f5a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.454295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38da84-6912-40ca-aa5e-0260f7119f5a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.454569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38da84-6912-40ca-aa5e-0260f7119f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.468282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:22.469959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 344/693] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:22.474870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1d38da84-6912-40ca-aa5e-0260f7119f5a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:22.475529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1d38da84-6912-40ca-aa5e-0260f7119f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:22.476175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:22.553516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 350/694] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:22.610117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7898490-b6cc-434c-a23b-7826a7c4aaa5 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:22.638022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:22.646762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:22.647679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.648318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.648585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.649299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] RBAC: Adding query filter params (role.id=a499fa21d23942afae2d8c3066f53b41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=b840fff8907e4931af6a5133eda97e0d) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:22.649626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:22.651040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55b5416-2776-42d4-a3db-55be6192a5fa None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:22.657480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 345/695] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:22 2022] GET /identity/v3/role_assignments?user.id=b840fff8907e4931af6a5133eda97e0d&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=a499fa21d23942afae2d8c3066f53b41 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:22.695237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7236c3c-0b55-4a24-bc81-8dbbed87725f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.695880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7236c3c-0b55-4a24-bc81-8dbbed87725f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.696352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7236c3c-0b55-4a24-bc81-8dbbed87725f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.698161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 351/696] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:22.703737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0cb9d-8ae2-4ac0-807f-0efa1990b3c7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.703918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.704000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.704102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0cb9d-8ae2-4ac0-807f-0efa1990b3c7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.704338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c0cb9d-8ae2-4ac0-807f-0efa1990b3c7 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.704435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.705751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 346/697] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:22.711275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.711572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.711812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.725214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:22.726515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:22.731861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:22.733741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:22.808103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 352/698] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:22.808618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 347/699] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:22.837911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:22.911261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:22.953169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985119b-2fae-43a1-9b26-8c404af9e643 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:22.988676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:22.989222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:22.989525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:22.989794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:22.990370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:22.991725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:23.001299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:23.002296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:23.003127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 353/700] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:23.008486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.093029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.119430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e552115-cf23-4aec-8d87-02fcec1910d2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.144259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.157251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.157886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.158164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.158433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.159053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.159382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.160764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.165760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 348/701] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.166439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.214275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204a33aa-074a-4024-b2c5-c43fb47f1b7c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.245025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.252018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.252655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.252935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.253206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.253815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.255189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:23.265228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:23.266345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:23.267177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:23.271482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 354/702] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:23.272512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.289505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b36abca-fb8a-4574-9ab7-aada3c53084e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.332639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.333438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.333714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.333979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.334547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.335931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:23.344929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:23.345990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:23.346836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 349/703] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:23.347685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.383735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a89636d-75df-4600-8b17-190b5acac3c7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.400931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.421477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.422201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.422538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.422879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.423506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.423801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.425153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.430668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 355/704] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.436493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.454242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc050462-e850-42cd-ac64-b4e111339ae9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.490293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.491402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.491685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.491953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.492842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.493145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.494777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.497212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.505449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 350/705] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.506308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.546459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b2013d-f46e-4d0d-857a-2334f6cfb4f5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.581568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.582172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.582452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.582725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.583349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.583654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.585115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.590226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 356/706] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.590901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.616978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed6127e-a6e2-4896-a626-903aab6e5526 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.635048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.656302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.656913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.657196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.657502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.658201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] RBAC: Adding query filter params (interface=public, service_id=ac96e396eeb94f2bbb1e5c6346680300, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.658514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.659935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.665027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 351/707] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/endpoints?service_id=ac96e396eeb94f2bbb1e5c6346680300&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.665826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.710036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8bf49-1a33-420b-97d9-8ecf48a378a1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.734176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.751776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.752314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.752598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.752871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.753525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.753835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.755212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.760364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 357/708] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.761173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.782291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfdf100-5bc2-43a7-bfc7-b75df38df92b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.818349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.818951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.819233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.819504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.820533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:23.820840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.822248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.827685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 352/709] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:23.828530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:23.880932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3cb1d-745d-4446-a2cb-08c83429f1fb None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:23.898871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:23.923275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:23.923897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:23.924193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:23.924455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:23.925060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:23.926489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:23.941790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 358/710] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:19:23.976667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a96d78-5073-4bde-922e-49d0046a8f4d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:24.018893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:24.019434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.019715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.019986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.020717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] RBAC: Adding query filter params (interface=public, service_id=d496c14d64504102a513d9ab263c9b12, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:24.021029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:24.022505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fe29ba-b678-4b14-85d7-21f976cd965e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:24.029071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 353/711] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:23 2022] GET /identity/v3/endpoints?service_id=d496c14d64504102a513d9ab263c9b12&interface=public®ion_id=RegionOne => generated 194 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:24.035033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:24.101706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:24.157302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5d991-fed2-409b-b50c-08254262c925 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:24.197473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6bbee-3e17-4bf6-b541-d95b17a8e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:24.198086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6bbee-3e17-4bf6-b541-d95b17a8e140 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.198365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6bbee-3e17-4bf6-b541-d95b17a8e140 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.198637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6bbee-3e17-4bf6-b541-d95b17a8e140 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.199240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6bbee-3e17-4bf6-b541-d95b17a8e140 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:24.200608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6bbee-3e17-4bf6-b541-d95b17a8e140 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:24.207880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 359/712] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:24 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:19:24.947221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bdff7-7572-4b48-89be-27bf136dc4d6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.947858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bdff7-7572-4b48-89be-27bf136dc4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.947858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bdff7-7572-4b48-89be-27bf136dc4d6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.949360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 354/713] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:24.955354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:24.955763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:24.955931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:24.975288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:24.976477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:25.046936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 360/714] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:25.320083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bdff7-7572-4b48-89be-27bf136dc4d6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:25.428890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bdff7-7572-4b48-89be-27bf136dc4d6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:25.435930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:25.466358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:25.466982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:25.467827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:25.468095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] PATH_INFO: `/v3/services/image` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:25.468665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:25.470057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:25.479046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:25.480037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:25.480886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 355/715] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Oct 12 12:19:25 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:25.486409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:25.534783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:25.590930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994c2cbf-ba46-45bf-a8e3-cc1cf071f1b1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:25.626418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:25.627033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:25.627308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:25.627580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:25.628211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] RBAC: Adding query filter params (name=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:25.628514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:25.629940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce860e4-f20a-4997-81da-cfd4bd965dcd None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:25.635033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 361/716] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:25 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:25.641221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:25.744117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba42fe9-e598-4872-b3f4-414775d545da None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:25.778976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:25.779686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:25.780612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:25.780976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:25.781641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] RBAC: Adding query filter params (type=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:25.781996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:25.783376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac190-4919-4004-981b-26775132a553 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:25.788373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 356/717] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:25 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:26.122846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c94a2-76c5-4524-ba65-0c8793674df2 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.123431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c94a2-76c5-4524-ba65-0c8793674df2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.123431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c94a2-76c5-4524-ba65-0c8793674df2 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.125564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 362/718] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:26.131390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85498-0152-4719-b0fb-83fbf9a53db8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.132036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85498-0152-4719-b0fb-83fbf9a53db8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.132036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a85498-0152-4719-b0fb-83fbf9a53db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.150820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c8a85498-0152-4719-b0fb-83fbf9a53db8 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:26.152030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c8a85498-0152-4719-b0fb-83fbf9a53db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:26.180637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:26.222707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 357/719] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:26.237955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff507e-bf50-4dd4-9c50-691e9fac0734 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.238232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff507e-bf50-4dd4-9c50-691e9fac0734 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.238501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eff507e-bf50-4dd4-9c50-691e9fac0734 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.240067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 363/720] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:26.245219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c9b6f-cc33-464f-8a68-e6fc72589124 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.245934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c9b6f-cc33-464f-8a68-e6fc72589124 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.246350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8c9b6f-cc33-464f-8a68-e6fc72589124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.267780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6c8c9b6f-cc33-464f-8a68-e6fc72589124 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:26.269204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6c8c9b6f-cc33-464f-8a68-e6fc72589124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:26.339176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 358/721] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:26.493633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79613425-8a4d-425f-a3be-2809b22c408c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.493914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79613425-8a4d-425f-a3be-2809b22c408c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.494188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79613425-8a4d-425f-a3be-2809b22c408c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.496018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 364/722] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:26.502099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79613425-8a4d-425f-a3be-2809b22c408c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:26.503502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.503783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.504053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.523417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:26.524624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:26.605484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 359/723] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:26.619570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:26.626391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79613425-8a4d-425f-a3be-2809b22c408c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:26.636044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:26.653680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:26.663769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:26.664376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.664653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.664921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.665613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:26.666980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:26.684127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 365/724] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:26 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:26.747693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5531a2-2ada-46be-8ad2-e53e5cfe1962 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:26.797979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:26.799008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.799486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.799925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.800694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:26.802344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:26.812722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:26.813809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 12 12:19:26.814630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:26.817244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 360/725] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:26.827079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:26.894340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:26.953982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5700e930-4bc4-4ef5-b38f-59d725da1ddc None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:26.989652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:26.990266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:26.990546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:26.990817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:26.991451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:26.991753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:26.993122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:26.999355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 366/726] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:26 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:27.006580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.015434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.033814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37485fca-09c9-47b1-9877-60111c9a7b26 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.049687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.082153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.083696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.084155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.084825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.085853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.088184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:27.100924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:27.102044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:27.102860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:27.106050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 361/727] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:27.112009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.133429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b22ad-0aac-4a78-88a1-12fa2277eefe None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.173773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.174762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.175261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.175735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.176557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:27.177058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.179658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.185889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 367/728] 10.222.0.17 () {56 vars in 1114 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:27.191723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.243449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c2c11-d329-4c38-acee-637aa10752ec None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.283174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.284153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.284692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.285200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.286040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:27.286535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.288144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.297908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.298546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 362/729] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:27.299663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f4974-fff2-4783-badd-ef1890e4f36c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.299963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.315591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:27.339670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.340309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.340585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.340856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.341554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.343493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.373829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 368/730] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:27.412864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248f0c3-f152-419a-b5c8-4e9961b99821 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.457758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.458782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.459257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.459723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.460571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:27.461064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.462746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd286cb4-1844-4edc-a877-4bb07be6139d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.469474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 363/731] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:27.476344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:27.589575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3622d157-4648-46df-bb89-850eaa685273 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:27.626703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afec3f-6a48-42f8-ae1c-8e571f3cd453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:27.627384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afec3f-6a48-42f8-ae1c-8e571f3cd453 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:27.627651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afec3f-6a48-42f8-ae1c-8e571f3cd453 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:27.627950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afec3f-6a48-42f8-ae1c-8e571f3cd453 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:27.628681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afec3f-6a48-42f8-ae1c-8e571f3cd453 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:27.630107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afec3f-6a48-42f8-ae1c-8e571f3cd453 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:27.655759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 369/732] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:27 2022] POST /identity/v3/endpoints => generated 349 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:28.030778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b543c5d-fe53-48d6-b572-8689140043a0 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:28.031294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b543c5d-fe53-48d6-b572-8689140043a0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:28.031683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b543c5d-fe53-48d6-b572-8689140043a0 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:28.033419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 364/733] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:28.038912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926cd1-00df-4f4f-b0a3-d0471b8f682e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:28.039206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926cd1-00df-4f4f-b0a3-d0471b8f682e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:28.039486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c926cd1-00df-4f4f-b0a3-d0471b8f682e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:28.058614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6c926cd1-00df-4f4f-b0a3-d0471b8f682e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:28.059839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6c926cd1-00df-4f4f-b0a3-d0471b8f682e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:28.062900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:28.139950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 370/734] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:28.423120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b543c5d-fe53-48d6-b572-8689140043a0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:28.495245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:28.531547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b543c5d-fe53-48d6-b572-8689140043a0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:28.565499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5209a191-d6a8-4c0e-9605-4eec85097aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:28.566114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a191-d6a8-4c0e-9605-4eec85097aee None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:28.566387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a191-d6a8-4c0e-9605-4eec85097aee None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:28.566653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5209a191-d6a8-4c0e-9605-4eec85097aee None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:28.567318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209a191-d6a8-4c0e-9605-4eec85097aee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:28.568715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5209a191-d6a8-4c0e-9605-4eec85097aee None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:28.590629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 365/735] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:28 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:28.979182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5872c1-3a7d-4e71-a91c-399ac96eca19 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:28.979575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5872c1-3a7d-4e71-a91c-399ac96eca19 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:28.979760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5872c1-3a7d-4e71-a91c-399ac96eca19 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:28.981339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 371/736] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:28.986672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:28.986974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:28.987235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.005672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:29.006859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:29.080558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 366/737] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:28 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:29.356480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5872c1-3a7d-4e71-a91c-399ac96eca19 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:29.464390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5872c1-3a7d-4e71-a91c-399ac96eca19 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:29.506452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:29.507485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.507970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.508462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] PATH_INFO: `/v3/services/compute` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.509260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:29.510886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:29.513366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:29.521961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:29.522976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:29.523825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 372/738] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:29.529104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:29.599311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:29.642274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d089ac46-37a9-400c-911f-4cb4151f0179 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:29.677717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:29.678372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.678651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.678919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.679599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:29.679901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:29.681321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581ffeb-2c79-4504-93c0-ea4f662fc342 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:29.686805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 367/739] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:29.692107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:29.696309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d574be2-f633-4f83-bf31-6c1683deaf9a None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.696530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d574be2-f633-4f83-bf31-6c1683deaf9a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.696737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d574be2-f633-4f83-bf31-6c1683deaf9a None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.698299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 368/740] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:29.706875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.707158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.707436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.725719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:29.726903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:29.807719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766e7147-d3af-4c70-a3ff-362cc90426ae None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:29.813751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 369/741] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:29 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:29.849332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:29.850326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.850799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.851365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.852199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:29.852691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:29.854285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7723cfa2-4a97-490f-b064-cda9f3e83bcc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:29.860564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 373/742] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:29.866177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:29.922487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:29.941014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f52bd-ee13-47a9-9da9-99a7d832018f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.941677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f52bd-ee13-47a9-9da9-99a7d832018f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.941949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f52bd-ee13-47a9-9da9-99a7d832018f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.943440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 374/743] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:29.950629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:29.951147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:29.952190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:29.971500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:29.972980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:29.976968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b426962d-a1e1-464b-8159-07e63caa47f1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.016639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.018001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.018281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.018552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.019247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] RBAC: Adding query filter params (interface=public, service_id=dc4d9ad63e9340d3b1d8b17c19ff1fe8, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:30.019554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.020942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.030569 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 370/744] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:29 2022] GET /identity/v3/endpoints?service_id=dc4d9ad63e9340d3b1d8b17c19ff1fe8&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:30.032447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.047869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.060157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 375/745] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:29 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:30.096219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.148695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a9a50d-c6ee-4912-b001-0dea21b720b3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.166984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.186944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.187650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.187992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.188355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.189023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.190494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.197524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 371/746] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/services => generated 1957 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:19:30.223376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e28ce8-7f36-4171-915d-1a2e96ae1abb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.274990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.275947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.276439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.276884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.277787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.279587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.282843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING keystone.server.flask.common [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 12 12:19:30.305646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.sql.core [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:30.305646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:19:30.305646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [parameters: {'id': '7d96fb223afc42298dfb2775fd1896c1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 12:19:30.305646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d96fb223afc42298dfb2775fd1896c1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d96fb223afc42298dfb2775fd1896c1. Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:30.314984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 12:19:30.315838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application yield resource Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise self.value Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application session.commit() Oct 12 12:19:30.316858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:19:30.317815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:19:30.318747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:30.319927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [parameters: {'id': '7d96fb223afc42298dfb2775fd1896c1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.320901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:30.321767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7d96fb223afc42298dfb2775fd1896c1. Oct 12 12:19:30.322725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.325079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 376/747] 10.222.0.17 () {60 vars in 1140 bytes} [Wed Oct 12 12:19:30 2022] POST /identity/v3/projects => generated 262 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:30.331347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.375604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.434407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.457932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e925c82-90b2-4cd2-b360-8df0872a44cb None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.496136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54522bd8-ac1a-4e55-b970-0ac0c4e454e9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.496784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.497600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.498146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.499382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.506719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.508453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:30.521908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:30.522994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:30.524080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:30.526775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 372/748] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:30.533337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.538470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.539034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.539409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.539775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.540549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.542262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:30.552955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:30.554213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:30.555042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 377/749] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:30.566327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.588216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.657938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e52b6-b42e-4c5a-8650-e0faf52fd186 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.680017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcd2b8-e1fb-4370-8777-b8abe8a74bb6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.695646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.696704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.697408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.697688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.697960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.698993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:30.699465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.701471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.724450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.725227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.725649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.726060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.726805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:30.727235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.728782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf0bfc-0bbe-4e99-b34e-4c8e04ed01e9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.730431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 373/750] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:30.739186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 378/751] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:30.745478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.830408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a569c-7283-4d4b-bf04-51b2433e3f50 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.830599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a569c-7283-4d4b-bf04-51b2433e3f50 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.830869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a569c-7283-4d4b-bf04-51b2433e3f50 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.832521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 379/752] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:30.838094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.838378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.838656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.859703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:30.861477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:30.865631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:30.868500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9824116-97e2-4379-9383-605041f08f62 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:30.909451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:30.910358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:30.910731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:30.911098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:30.911934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:30.912304 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:30.913787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:30.919149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 374/753] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:30.925993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:30.959126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 380/754] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:30 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:31.050376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb8323e-64b7-4c6c-8ca5-461922413d9d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.090975 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.091499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.091775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.092636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.093827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] RBAC: Adding query filter params (interface=admin, service_id=d496c14d64504102a513d9ab263c9b12, region_id=RegionOne) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:31.094143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.095557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.101561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 375/755] 10.222.0.17 () {56 vars in 1249 bytes} [Wed Oct 12 12:19:30 2022] GET /identity/v3/endpoints?service_id=d496c14d64504102a513d9ab263c9b12&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:31.110417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.224239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595eee7-6884-47a6-b8f6-2691c28dcb21 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.231969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.233551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.269131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.269890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.270263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.270651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.271352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.272838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.283902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 381/756] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:19:31.288861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.378403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e8866-5dd2-4f83-b26b-3621bceb09e3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.418541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214665-d637-494f-bf8d-909033b72541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.419207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214665-d637-494f-bf8d-909033b72541 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.419493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214665-d637-494f-bf8d-909033b72541 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.419771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38214665-d637-494f-bf8d-909033b72541 None admin] PATH_INFO: `/v3/services/image` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.420920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38214665-d637-494f-bf8d-909033b72541 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.422413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38214665-d637-494f-bf8d-909033b72541 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-38214665-d637-494f-bf8d-909033b72541 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:31.432260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:31.433353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:31.434495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 376/757] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/services/image => generated 86 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:31.442421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.553073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725b23a-613f-47ee-b47e-8f865627fd55 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.588660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.589268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.589567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.589837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.590456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] RBAC: Adding query filter params (name=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:31.590758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.592214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.597570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 382/758] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:31.603428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214665-d637-494f-bf8d-909033b72541 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.703421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.712031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38214665-d637-494f-bf8d-909033b72541 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.748615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.749225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.749533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.749807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.750433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] RBAC: Adding query filter params (type=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:31.750737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.752107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.757546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 377/759] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:31.763409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:31.804151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:31.875029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a1658-7fec-4201-bfa6-6a6110b22380 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:31.910359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:31.910961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:31.911237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:31.911506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:31.912192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] RBAC: Adding query filter params (interface=public, service_id=6a59373a0b184b538a850a6aef86e4e9, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:31.912497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:31.913886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ac7208-4a58-44fc-a504-ae7fe4d0eeec None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:31.919207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 383/760] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/endpoints?service_id=6a59373a0b184b538a850a6aef86e4e9&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:31.925022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.032422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f50de-7e23-4315-9c80-d23593d1e462 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:32.066940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594b2f3-9dfa-4b64-88cb-76687bddb17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:32.067548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b2f3-9dfa-4b64-88cb-76687bddb17c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.067832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b2f3-9dfa-4b64-88cb-76687bddb17c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.068105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594b2f3-9dfa-4b64-88cb-76687bddb17c None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.068704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594b2f3-9dfa-4b64-88cb-76687bddb17c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:32.070102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a594b2f3-9dfa-4b64-88cb-76687bddb17c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:32.077555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 378/761] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:19:31 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:19:32.492858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a740536-e636-43b1-9cb2-6fab36100f41 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.493391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a740536-e636-43b1-9cb2-6fab36100f41 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.493495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a740536-e636-43b1-9cb2-6fab36100f41 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.495174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 384/762] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:32.501100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:32.501423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:32.501700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:32.524587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:32.525822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:32.540961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:32.602483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 379/763] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:32 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:32.871215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a740536-e636-43b1-9cb2-6fab36100f41 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:32.978690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:32.984242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a740536-e636-43b1-9cb2-6fab36100f41 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.024922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.025580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.025854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.026118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.026688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.028042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c54245d7-0def-404c-ac98-b47fcc3bc88f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:33.037158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:33.038176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.039627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 385/764] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:33.044123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.155562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925d7b18-216a-4dcc-aac4-7f2524510aeb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.191349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.191956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.192236 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.192509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.193137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:33.193468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.194846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.196736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e64e5-4e16-4257-afaa-99293c100f67 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.197082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e64e5-4e16-4257-afaa-99293c100f67 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.197414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9e64e5-4e16-4257-afaa-99293c100f67 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.199157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 386/765] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:33.200048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 380/766] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:33.205948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.206230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.206458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.206927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.226321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:33.229259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:33.231102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:33.317999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1c3b6-9ab0-47f3-af04-959a37c9615f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.337464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 387/767] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:33 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:33.362201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.362919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.363201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.363474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.364098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:33.364399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.365816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff37c87-4172-434a-b752-15d73db378c3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.371406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 381/768] 10.222.0.17 () {56 vars in 1112 bytes} [Wed Oct 12 12:19:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:33.377902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.477553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:33.491222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbc205-c265-4a1e-ae7c-e35fdd400a8b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.513496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0eb64-6aa5-4b9f-95ac-5b95682cb3c2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.513977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0eb64-6aa5-4b9f-95ac-5b95682cb3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.514137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0eb64-6aa5-4b9f-95ac-5b95682cb3c2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.515601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 382/769] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:33.520598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.520875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.521140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.527121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.527921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.528999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.529294 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.530165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.531541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.538981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:33.540255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:33.557357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 388/770] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:33.628381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 383/771] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:33 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:33.664886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.777281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35b0b99-9648-417d-9ca4-cae31a561e1b None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:33.813935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:33.814803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:33.815229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:33.815668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:33.816464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:33.818074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:33.836341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.sql.core [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 12:19:33.836341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 12:19:33.836341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [parameters: {'user_id': '4e628e027d96444ebeaf611affa4027a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 12:19:33.836341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81424) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:33.844834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:33.845801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application yield resource Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise self.value Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application session.commit() Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:19:33.846797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:19:33.847715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:33.848621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:33.850511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [parameters: {'user_id': '4e628e027d96444ebeaf611affa4027a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:33.852531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 12 12:19:33.854584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:33.856757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 389/772] 10.222.0.17 () {60 vars in 1131 bytes} [Wed Oct 12 12:19:33 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:33.857740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.858075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:33.906648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:33.982073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50259c48-8729-44d5-952d-034c89c25b69 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.024004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.024923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.025324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.025698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.033814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.035969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.037049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:34.056101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:34.057564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 12 12:19:34.058728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:34.063796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 384/773] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:34.064877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:34.131022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:34.159732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd71359-c334-498a-9b0a-fc5a4d520e08 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.179006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b591bc67-d05c-4966-897e-7f01f2eb886a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.198158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.198961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.199299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.199636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.200411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.202797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:34.212000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:34.213227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 12 12:19:34.214251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:34.215072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 390/774] 10.222.0.17 () {56 vars in 1126 bytes} [Wed Oct 12 12:19:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:34.220917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:34.228453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.229181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.229556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.229900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.230700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:34.231079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.233921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e645f3b-394f-48f3-a0b8-0c772836042c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.292500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 385/775] 10.222.0.17 () {56 vars in 1101 bytes} [Wed Oct 12 12:19:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:34.313164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:34.343939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd7c36-108f-48e6-9785-56879ac78ee2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.353033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5557ede-752d-45c2-abb1-8c7512e260b6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.353033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5557ede-752d-45c2-abb1-8c7512e260b6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.353299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5557ede-752d-45c2-abb1-8c7512e260b6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.359839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 386/776] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:34.361683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.362046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.362776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.383647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:34.385035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:34.390459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.391388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.391849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.392421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.393351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:34.393851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.396334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.409061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 391/777] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:34.415076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:34.477745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 387/778] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:34 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:34.539085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6c39d-53b3-4cbc-b3ea-d188381718b4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.575199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.575740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.576015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.576339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.577499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:34.577789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.579161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.584639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 392/779] 10.222.0.17 () {56 vars in 1122 bytes} [Wed Oct 12 12:19:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:34.590373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:34.648485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:34.698464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ea581f-6cb8-4cb9-afc5-665222871bee None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.732260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.732941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.733222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.733532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.734205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.735589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.753329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 388/780] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:34.753517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:34.757507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:34.867766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed565f-df3c-49e7-8a96-3f488b7525bf None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:34.910810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:34.911456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:34.911731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:34.911997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] PATH_INFO: `/v3/services/image` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:34.912572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:34.913979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:34.924026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:34.925405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:34.926726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 393/781] 10.222.0.17 () {56 vars in 1105 bytes} [Wed Oct 12 12:19:34 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:34.930881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:35.036098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1101f9e5-8706-47c3-bf86-7f083abaae3f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:35.049422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:35.072397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:35.072994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.073260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.073559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.074180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] RBAC: Adding query filter params (name=image) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:35.074478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:35.075821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:35.080778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 389/782] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:34 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:35.086720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:35.150544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:35.193061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e3d9bb-46ea-4923-9065-4b0a48fb6e32 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:35.229228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:35.229859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.230139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.230407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.231031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] RBAC: Adding query filter params (type=image) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:35.231332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:35.233238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f00f6d-77d7-4a21-9b4b-043fcb449687 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:35.238373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 394/783] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:35.243831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:35.353658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98739a97-4ba3-489b-9557-3e5f57250e72 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:35.387159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b09af-ed65-4837-9249-9d46a21321c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:35.387766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b09af-ed65-4837-9249-9d46a21321c7 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.388047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b09af-ed65-4837-9249-9d46a21321c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.388320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b09af-ed65-4837-9249-9d46a21321c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.389031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b09af-ed65-4837-9249-9d46a21321c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:35.390547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b09af-ed65-4837-9249-9d46a21321c7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:35.417429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 390/784] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:19:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:35.911188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fb40d-e906-4f91-89f7-baff6795e15d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.911727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fb40d-e906-4f91-89f7-baff6795e15d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.911727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fb40d-e906-4f91-89f7-baff6795e15d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.913192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 395/785] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:35.918134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:35.918426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:35.918700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:35.937238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:35.938451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:35.965669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:36.018156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 391/786] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:36.289078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fb40d-e906-4f91-89f7-baff6795e15d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:36.392859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fb40d-e906-4f91-89f7-baff6795e15d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:36.418388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:36.427933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:36.428561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.428842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.429116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.439412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:36.444296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:36.445207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:36.446349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:36.449611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 396/787] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:19:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:36.451997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:36.551064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ed1cd-25d8-4c38-9f17-d1adb4ceaa63 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:36.584767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:36.585399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.585683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.585955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.586620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:36.586921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:36.588305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:36.601360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 392/788] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Oct 12 12:19:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:36.602638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:36.705613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c3e43d-3211-4dad-8341-8333756321e7 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:36.739718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:36.740345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.740625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.740896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.751246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:36.752695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:36.767052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:36.768017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:36.769534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:36.769534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:36.769534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:36.769534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 397/789] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:36.773792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:36.798711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:36.883174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a2c1f-5021-4c73-a511-cbb068553520 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:36.916323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:36.916934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.917215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.917517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.918160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:36.918463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:36.919843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cc1f36-4d2a-4b35-a109-9987f27e7e7a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:36.932397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 393/790] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:36.937901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:36.946972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1426-43dc-4153-8bb3-8f6301075479 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.947261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1426-43dc-4153-8bb3-8f6301075479 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.947535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b1426-43dc-4153-8bb3-8f6301075479 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.948846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 394/791] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:36.954031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:36.954312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:36.954580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:36.973663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:36.974849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:36.980495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.045345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 395/792] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:37.053240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11c33bd-2343-416b-9401-458b46cb821e None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.093028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.093924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.094222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.094495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.101914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.103344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:37.115066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:37.116003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:19:37.117033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:37.118265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 398/793] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Oct 12 12:19:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:37.122208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.134266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.225171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df9585-ec01-4e39-8517-56c765cc1451 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.259908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.260525 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.260807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.261079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.261833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:37.262138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.265460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.288296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 396/794] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Oct 12 12:19:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:37.289535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.320612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.388227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.393341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55026c3-5a9d-4a11-91e9-c964fe598a95 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.400118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.430175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.430807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.431088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.431360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.436847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7538a63-c9d2-45f8-9862-b83045a60468 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.442687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.444133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:37.458918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:37.459889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:37.460762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:37.460762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:37.460762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:37.460762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 399/795] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:37.465611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.475324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.475954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.476233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.476506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.477643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.479078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.480679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 397/796] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:37.488033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.585695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c722e16-b4f7-4fa5-b641-547eb2dcd4d8 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.593437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7088fea3-7deb-4698-9206-fa5e8e10907f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.618891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.619507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.619779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.620049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.620938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:37.621239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.622628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b25949-38ad-4ab4-9a0d-29837ffa01bc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.629843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dc21ff-ed30-425a-adb7-a4cd319b7e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.630443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc21ff-ed30-425a-adb7-a4cd319b7e13 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.630719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc21ff-ed30-425a-adb7-a4cd319b7e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.630987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dc21ff-ed30-425a-adb7-a4cd319b7e13 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.631647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc21ff-ed30-425a-adb7-a4cd319b7e13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.633040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dc21ff-ed30-425a-adb7-a4cd319b7e13 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.640297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 400/797] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:19:37.640297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064a4b-f6a6-41b2-9673-ce51c6556762 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.640297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064a4b-f6a6-41b2-9673-ce51c6556762 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.640297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064a4b-f6a6-41b2-9673-ce51c6556762 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.640297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 401/798] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:37.640737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.641375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064a4b-f6a6-41b2-9673-ce51c6556762 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.654078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.662549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 398/799] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:37 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:19:37.663936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.664230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.664502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.684656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:37.686468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:37.772865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 399/800] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:37.776352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064a4b-f6a6-41b2-9673-ce51c6556762 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.815526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.816170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.816452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.816722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.824429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.825942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:37.838170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:37.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:37.840156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 402/801] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:37.845474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:37.953461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d3ffd-8bce-4332-9345-013eae296a05 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:37.972362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:37.990758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:37.991362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:37.991634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:37.991899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:37.992718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:37.993022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:37.994477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:38.007066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 400/802] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:38.013125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:38.044718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:38.087810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:38.122884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a542cdd-2454-4936-8237-5c12c8584f03 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:38.151644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdbd9e-67d3-453d-ba8f-87cc3004c01c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:38.159338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:38.159999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.160277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.160785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/5e2c96c17470485f90e95e3e5bb87ffa/roles/a499fa21d23942afae2d8c3066f53b41` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.162731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=5e2c96c17470485f90e95e3e5bb87ffa, role_id=a499fa21d23942afae2d8c3066f53b41)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:38.164164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:38.185351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 403/803] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:38 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/5e2c96c17470485f90e95e3e5bb87ffa/roles/a499fa21d23942afae2d8c3066f53b41 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:38.203402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:38.204131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.204357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.204626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.205335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:38.206815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:38.216598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.sql.core [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:38.216598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:19:38.216598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [parameters: {'id': '737ba7e052c8440db0dc4ef31c90528c', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:19:38.216598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81425) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:38.233065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 12 12:19:38.234044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application yield resource Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application next(self.gen) Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.rollback() Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.force_reraise() Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise self.value Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application session.commit() Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 12 12:19:38.235181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._prepare_impl() Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.session.flush() Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._flush(objects) Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application compat.raise_( Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application flush_context.execute() Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rec.execute(self) Oct 12 12:19:38.236900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 12 12:19:38.238150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result = self._query(query) Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application conn.query(q) Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application result.read() Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 12 12:19:38.239132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [parameters: {'id': '737ba7e052c8440db0dc4ef31c90528c', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 12 12:19:38.240614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:38.243364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:38.244507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 401/804] 10.222.0.17 () {60 vars in 1137 bytes} [Wed Oct 12 12:19:38 2022] POST /identity/v3/domains => generated 190 bytes in 191 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:38.245432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:38.361080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a08d3ec-3158-49e5-8413-12189a67ca00 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:38.391077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:38.398819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:38.399518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.399862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.400198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.412028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:38.413576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-31cb542d-8fed-45f9-a255-221fa260c3be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:38.428671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:38.429941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:38.431091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:38.431091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:38.431091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:38.437083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 404/805] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:38.438580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:38.484846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:38.546729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f709aad-25ef-43a1-b34a-e3f55a660fc1 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:38.580449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:38.581052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:38.581354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:38.581633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:38.582265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:38.582568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:38.583969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5a105-b3f2-4abb-8361-4683af0bbf3f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:38.595735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 402/806] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:39.893360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372017d-ff57-48d1-884e-3a9d6e4dde22 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:39.894229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372017d-ff57-48d1-884e-3a9d6e4dde22 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:39.894510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372017d-ff57-48d1-884e-3a9d6e4dde22 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:39.895854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 405/807] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:39.901296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeb474-b647-49c1-9263-9d431d073cd6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:39.901726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeb474-b647-49c1-9263-9d431d073cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:39.901876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaeb474-b647-49c1-9263-9d431d073cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:39.919500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5eaeb474-b647-49c1-9263-9d431d073cd6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:39.920727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5eaeb474-b647-49c1-9263-9d431d073cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:39.986450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 403/808] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:40.262019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372017d-ff57-48d1-884e-3a9d6e4dde22 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:40.339020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1de48-e4ad-4893-a39f-09513b6dc845 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:40.339303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1de48-e4ad-4893-a39f-09513b6dc845 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:40.339577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f1de48-e4ad-4893-a39f-09513b6dc845 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:40.340902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 404/809] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:40.345783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:40.346062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:40.346334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:40.366006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:40.366634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372017d-ff57-48d1-884e-3a9d6e4dde22 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:40.367197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:40.393789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:40.401927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:40.402548 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:40.402828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:40.403098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:40.404201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:40.406159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:40.407705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 406/810] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:40.413595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:40.466912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:40.519211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f4ed1-e474-4b47-b303-e7a4fdfea3f6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:40.552368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:40.552577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 405/811] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:40.552970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:40.553244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:40.553540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:40.554182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:40.555603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:40.577573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 407/812] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:40 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:19:40.827633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:40.925293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bcc332-93f5-4944-8223-3e1c2c901d07 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:40.959747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:40.960366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:40.960638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:40.960900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:40.971145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:40.975813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:40.976904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:40.978023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 406/813] 10.222.0.17 () {56 vars in 1120 bytes} [Wed Oct 12 12:19:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:40.985387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:41.081888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d47262-9115-480d-90e1-c2d92a31ab0d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:41.088718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:41.116379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:41.116981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:41.117253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:41.117546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:41.118192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:41.118487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:41.119841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:41.128688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 408/814] 10.222.0.17 () {56 vars in 1115 bytes} [Wed Oct 12 12:19:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:41.134929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:41.159508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:41.231868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543fab9a-339f-46ee-83d9-77021593e3f9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:41.270042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:41.270667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:41.270939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:41.271205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:41.281923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:41.283355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:41.298003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:41.299075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:41.299961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:41.299961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:41.299961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:41.299961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 407/815] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:41.304729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:41.401735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd64d93-f563-4298-b616-d62129fce984 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:41.433492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:41.434083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:41.434354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:41.434622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:41.435247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:41.436079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:41.437641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:41.448863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 409/816] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:41.455313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:41.530106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:41.555395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e8dd-7ca4-47f8-b55f-00569aead231 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:41.587938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:41.588547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:41.588824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:41.589094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:41.596240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:41.597697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:19:41.609346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:41.610473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:41.611421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 408/817] 10.222.0.17 () {56 vars in 1093 bytes} [Wed Oct 12 12:19:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:41.616187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:41.643800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:41.712161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384143a-c6a9-414a-8a3d-b3f8318288c5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:41.744611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:41.745207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:41.745501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:41.746033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:41.746740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:41.747036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:41.749167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:41.766022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 410/818] 10.222.0.17 () {56 vars in 1133 bytes} [Wed Oct 12 12:19:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:41.771606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:41.867279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaa7b-7d35-42e6-9181-d4478226b76a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:41.899392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:41.899982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:41.900253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:41.900517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:41.903047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:41.966475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:41.968456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:42.061122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 409/819] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 12:19:42.066978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:42.148979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:42.164401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f81d2-8733-4ae5-b774-31d034a07111 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:42.197748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:42.198369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.198647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.198916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.208915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:42.210386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:19:42.224912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:42.225962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:19:42.226893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:19:42.226893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:19:42.226893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:42.226893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 411/820] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:42.231637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:42.271293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:42.336362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649492da-e8b5-423a-b8b3-ecb0b622f807 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:42.369197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:42.369819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.370100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.370372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.371263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:42.371560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:42.372924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:42.383857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 410/821] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:19:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:42.390011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:42.487090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a451e855-ce84-4d57-a999-9ad6d33773d0 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:42.520567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:42.521194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.521548 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.521827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.528264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:42.530280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:42.531959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:19:42.543116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:42.544131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:42.545156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 412/822] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:19:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:42.550331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:42.641617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:42.648906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a8306f-99c6-4db4-8b50-e8c11696500d None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:42.684849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:42.685491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.686306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.686574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.687262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:42.687563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:42.688990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6962c0-810a-471f-b2c0-50943f0e1e0a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:42.700797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 411/823] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:19:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:42.707341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:42.766226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6bb5-e6e6-4481-b277-b89b1c752c27 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.766454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6bb5-e6e6-4481-b277-b89b1c752c27 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.766732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ae6bb5-e6e6-4481-b277-b89b1c752c27 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.768053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 412/824] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:19:42.772877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.773161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.773467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.790861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:42.792043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:42.806166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175e3a7-8932-4ae8-b14d-ceae646a5a5d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:42.839729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:42.840326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:42.840605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:42.840883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:42.841628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] RBAC: Adding query filter params (role.id=a499fa21d23942afae2d8c3066f53b41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=5e2c96c17470485f90e95e3e5bb87ffa) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:42.841959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:42.843389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:42.848957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 413/825] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:42 2022] GET /identity/v3/role_assignments?user.id=5e2c96c17470485f90e95e3e5bb87ffa&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=a499fa21d23942afae2d8c3066f53b41 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:42.861492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 413/826] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:19:43.140040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.203616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.242848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc56dba3-5d90-4df2-86ca-5ffd8a61f2ea None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.277696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.278412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.278758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.279115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.290343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:43.295679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:43.296663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:19:43.297546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 414/827] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:19:43.303095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.390804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.407300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474509e5-357f-4202-a29f-1680e2e2aff9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.442281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.442885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.443161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.443430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.444081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:43.444381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.445778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.454765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 414/828] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.460927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.565638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0dfe60-d0c8-4f96-b764-aaa6e7a76e5a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.588682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:43.600607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.601229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.601538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.601815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] PATH_INFO: `/v3/users/a514bdb7a7094faca772a7d7a999bf74` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.615725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] RBAC: Authorizing `identity:get_user(user_id=a514bdb7a7094faca772a7d7a999bf74)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.617159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.618816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 415/829] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/users/a514bdb7a7094faca772a7d7a999bf74 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.625757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.725288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23e0a5-0605-47f0-b61e-697f042cdd6c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.758355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b215208d-7c59-4609-b69c-c8b77a3af165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.758966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215208d-7c59-4609-b69c-c8b77a3af165 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.759245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215208d-7c59-4609-b69c-c8b77a3af165 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.759518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b215208d-7c59-4609-b69c-c8b77a3af165 None admin] PATH_INFO: `/v3/projects/b94024e8a165476b8cb055d2fb4b8c48` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.760635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215208d-7c59-4609-b69c-c8b77a3af165 None admin] RBAC: Authorizing `identity:get_project(project_id=b94024e8a165476b8cb055d2fb4b8c48)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.762088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b215208d-7c59-4609-b69c-c8b77a3af165 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.764238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 415/830] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:43 2022] GET /identity/v3/projects/b94024e8a165476b8cb055d2fb4b8c48 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:43.770317 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:43.866188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dea5e4-3c42-4894-a153-a91fe61d9d4c None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:43.898849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c607606-78bf-4847-beb9-21674ca7b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:43.899478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c607606-78bf-4847-beb9-21674ca7b6f5 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:43.899757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c607606-78bf-4847-beb9-21674ca7b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:43.900030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c607606-78bf-4847-beb9-21674ca7b6f5 None admin] PATH_INFO: `/v3/projects/b94024e8a165476b8cb055d2fb4b8c48/users/a514bdb7a7094faca772a7d7a999bf74/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:43.901950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c607606-78bf-4847-beb9-21674ca7b6f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=b94024e8a165476b8cb055d2fb4b8c48, user_id=a514bdb7a7094faca772a7d7a999bf74, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:43.903360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c607606-78bf-4847-beb9-21674ca7b6f5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:43.915574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 416/831] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:43 2022] PUT /identity/v3/projects/b94024e8a165476b8cb055d2fb4b8c48/users/a514bdb7a7094faca772a7d7a999bf74/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:46.025715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896b82-9022-4eb6-a6a6-a44c9c0bc950 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:46.026249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896b82-9022-4eb6-a6a6-a44c9c0bc950 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:46.026249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2896b82-9022-4eb6-a6a6-a44c9c0bc950 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:46.027544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 416/832] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:46.032717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:46.033113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:46.033271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:46.051863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:46.053420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:46.086607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:46.120390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 417/833] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:46.391949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2896b82-9022-4eb6-a6a6-a44c9c0bc950 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:46.408063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:46.490183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2896b82-9022-4eb6-a6a6-a44c9c0bc950 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:46.521492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:46.522095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:46.522369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:46.522635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:46.533108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:19:46.537772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:19:46.538849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:19:46.539864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 417/834] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:19:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:19:46.545519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:46.640168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d649e52-9925-4ac3-b17f-49a3df2b49db None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:46.673660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:46.674191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:46.674467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:46.674738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:46.675398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:46.675698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:46.677051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:46.684834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 418/835] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:19:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:46.691215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:46.762732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:46.787462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0644fd0-aeff-4d8f-85c6-c37834ebce61 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:46.818848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:46.819450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:46.819729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:46.819998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] PATH_INFO: `/v3/users/a514bdb7a7094faca772a7d7a999bf74` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:46.821101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] RBAC: Authorizing `identity:get_user(user_id=a514bdb7a7094faca772a7d7a999bf74)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:46.822551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:46.824079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 418/836] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:46 2022] GET /identity/v3/users/a514bdb7a7094faca772a7d7a999bf74 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:46.830270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:46.860882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:46.927009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9edab2-2434-4723-86cc-1fcfcfdc2fb4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:46.959017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:46.959612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:46.959888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:46.960156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.024808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:47.026282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:47.054775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:47.122829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 419/837] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:19:47.129496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:47.229706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6826b4c9-b503-4ffa-95b1-9a942bb2015f None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:47.261957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d0343-83ca-443f-a8bd-c9b24c681745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:47.262623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0343-83ca-443f-a8bd-c9b24c681745 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.262856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0343-83ca-443f-a8bd-c9b24c681745 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.263126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d0343-83ca-443f-a8bd-c9b24c681745 None admin] PATH_INFO: `/v3/projects/b94024e8a165476b8cb055d2fb4b8c48` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.264263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d0343-83ca-443f-a8bd-c9b24c681745 None admin] RBAC: Authorizing `identity:get_project(project_id=b94024e8a165476b8cb055d2fb4b8c48)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:47.265724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d0343-83ca-443f-a8bd-c9b24c681745 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:47.267336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 419/838] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:47 2022] GET /identity/v3/projects/b94024e8a165476b8cb055d2fb4b8c48 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:47.273397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:47.369905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5227d-80ab-4680-a884-744b4d6c9546 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:47.397839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:47.402952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:47.403550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:47.403826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:47.404092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:47.405332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=b94024e8a165476b8cb055d2fb4b8c48, user.id=a514bdb7a7094faca772a7d7a999bf74) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:47.405651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:47.407069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef3527-ef5e-43ed-80de-8231dce40f6c None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:47.412645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 420/839] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:47 2022] GET /identity/v3/role_assignments?user.id=a514bdb7a7094faca772a7d7a999bf74&scope.project.id=b94024e8a165476b8cb055d2fb4b8c48&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:49.542684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7788-7d24-4115-a9bb-d06f871a5247 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.543185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7788-7d24-4115-a9bb-d06f871a5247 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.543185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e7788-7d24-4115-a9bb-d06f871a5247 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.544474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 420/840] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:49.549769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:49.550163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:49.550319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:49.568476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:49.569704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:49.634589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 421/841] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:49.900573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e7788-7d24-4115-a9bb-d06f871a5247 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:49.913329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.000713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e7788-7d24-4115-a9bb-d06f871a5247 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.032136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121c767-1d2b-47c9-a59d-7e3ca6a98df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.032737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121c767-1d2b-47c9-a59d-7e3ca6a98df2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.033013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121c767-1d2b-47c9-a59d-7e3ca6a98df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.033282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121c767-1d2b-47c9-a59d-7e3ca6a98df2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.034409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121c767-1d2b-47c9-a59d-7e3ca6a98df2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.035825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121c767-1d2b-47c9-a59d-7e3ca6a98df2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.037356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 421/842] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:50.042967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:50.139521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8348f90d-217b-4d2c-85c0-d947c3425387 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:50.164145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:50.173450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ca96-1198-4356-9486-babdbbdd67bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:50.174068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ca96-1198-4356-9486-babdbbdd67bf None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:50.174348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ca96-1198-4356-9486-babdbbdd67bf None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:50.174623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ca96-1198-4356-9486-babdbbdd67bf None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:50.175274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5ca96-1198-4356-9486-babdbbdd67bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:50.176703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5ca96-1198-4356-9486-babdbbdd67bf None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:50.196767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 422/843] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:19:50 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:19:52.378872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79ed2-21a5-4da3-8391-73026e55fe55 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:52.379421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79ed2-21a5-4da3-8391-73026e55fe55 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:52.379421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d79ed2-21a5-4da3-8391-73026e55fe55 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:52.380730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 422/844] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:52.385953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:52.386320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:52.386501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:52.405180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:52.406389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:52.470865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 423/845] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:52.739361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d79ed2-21a5-4da3-8391-73026e55fe55 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:52.817030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:52.838891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d79ed2-21a5-4da3-8391-73026e55fe55 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:52.871606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:52.872207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:52.872487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:52.872757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] PATH_INFO: `/v3/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:52.874756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] RBAC: Authorizing `identity:get_role(role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:52.876110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:52.877416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 423/846] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:52 2022] GET /identity/v3/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:52.883144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:52.980195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4822882-6fbe-4051-856e-27b4f51a4e51 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:52.993284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:53.015994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:53.016607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:53.016880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:53.017147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] PATH_INFO: `/v3/users/ee0e6021a9b44d8d83996df691f23d8b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:53.029868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0e6021a9b44d8d83996df691f23d8b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:53.031378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:53.032924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 424/847] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:52 2022] GET /identity/v3/users/ee0e6021a9b44d8d83996df691f23d8b => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:53.038857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:53.136003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da8178-1e2b-4850-9c75-97b5b869fede None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:53.168971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc4bdac-b656-45da-95d8-fe67cafebf77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:53.169608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4bdac-b656-45da-95d8-fe67cafebf77 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:53.169935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4bdac-b656-45da-95d8-fe67cafebf77 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:53.170209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc4bdac-b656-45da-95d8-fe67cafebf77 None admin] PATH_INFO: `/v3/projects/b94024e8a165476b8cb055d2fb4b8c48` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:53.171330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4bdac-b656-45da-95d8-fe67cafebf77 None admin] RBAC: Authorizing `identity:get_project(project_id=b94024e8a165476b8cb055d2fb4b8c48)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:53.172750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc4bdac-b656-45da-95d8-fe67cafebf77 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:53.174370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 424/848] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:53 2022] GET /identity/v3/projects/b94024e8a165476b8cb055d2fb4b8c48 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:53.180366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:53.275599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e307138-9038-4804-a666-650b1dcc5e01 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:53.307635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e20f30-55cc-48b1-86c0-1e9f7eb05d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:53.308269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e20f30-55cc-48b1-86c0-1e9f7eb05d20 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:53.308544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e20f30-55cc-48b1-86c0-1e9f7eb05d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:53.308815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e20f30-55cc-48b1-86c0-1e9f7eb05d20 None admin] PATH_INFO: `/v3/projects/b94024e8a165476b8cb055d2fb4b8c48/users/ee0e6021a9b44d8d83996df691f23d8b/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:53.311542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e20f30-55cc-48b1-86c0-1e9f7eb05d20 None admin] RBAC: Authorizing `identity:create_grant(project_id=b94024e8a165476b8cb055d2fb4b8c48, user_id=ee0e6021a9b44d8d83996df691f23d8b, role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:53.313026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e20f30-55cc-48b1-86c0-1e9f7eb05d20 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:53.326675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 425/849] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:19:53 2022] PUT /identity/v3/projects/b94024e8a165476b8cb055d2fb4b8c48/users/ee0e6021a9b44d8d83996df691f23d8b/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:19:55.442691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab989-4dce-4cb9-b04e-552e84b67fc6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.443237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab989-4dce-4cb9-b04e-552e84b67fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.443237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ab989-4dce-4cb9-b04e-552e84b67fc6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.444834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 425/850] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:55.449947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.450378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.450493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.468715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:55.469912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:55.492400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.535343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 426/851] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:55.800512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab989-4dce-4cb9-b04e-552e84b67fc6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:55.817269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:55.901385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ab989-4dce-4cb9-b04e-552e84b67fc6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:55.933489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:55.934123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:55.934397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:55.934664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] PATH_INFO: `/v3/roles/2b9a144dd3114f13a8fa1006f585fb12` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:55.937213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] RBAC: Authorizing `identity:get_role(role_id=2b9a144dd3114f13a8fa1006f585fb12)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:55.938581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:55.939862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 426/852] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/roles/2b9a144dd3114f13a8fa1006f585fb12 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:55.945718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:56.042924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130a9054-7c1e-4924-81bf-8e9e23df7d74 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:56.075787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:56.076948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:56.077225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:56.077522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] PATH_INFO: `/v3/users/ee0e6021a9b44d8d83996df691f23d8b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:56.078644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] RBAC: Authorizing `identity:get_user(user_id=ee0e6021a9b44d8d83996df691f23d8b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:56.080068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:56.081667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 427/853] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:19:55 2022] GET /identity/v3/users/ee0e6021a9b44d8d83996df691f23d8b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:56.087428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:56.166343 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:56.184200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1a691-1373-4c01-810d-dfaf77e604cc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:56.216359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:56.216985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:56.217414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:56.217686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:56.282823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:56.284289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:56.360577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:56.379558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 427/854] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:19:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:19:56.385563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:56.413273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:56.482683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee5113-dabf-4cf8-a737-30bd7b139de4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:56.515526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16571b67-5c6d-41d5-b6de-f154c93e7377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:56.516130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16571b67-5c6d-41d5-b6de-f154c93e7377 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:56.516407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16571b67-5c6d-41d5-b6de-f154c93e7377 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:56.516673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16571b67-5c6d-41d5-b6de-f154c93e7377 None admin] PATH_INFO: `/v3/projects/b94024e8a165476b8cb055d2fb4b8c48` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:56.517797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16571b67-5c6d-41d5-b6de-f154c93e7377 None admin] RBAC: Authorizing `identity:get_project(project_id=b94024e8a165476b8cb055d2fb4b8c48)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:56.519214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16571b67-5c6d-41d5-b6de-f154c93e7377 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:56.520797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 428/855] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:19:56 2022] GET /identity/v3/projects/b94024e8a165476b8cb055d2fb4b8c48 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:56.527030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:56.622617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b9714-de82-4c3f-a814-252632e09c7b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:56.655452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:56.656047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:56.656324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:56.656591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:56.657327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] RBAC: Adding query filter params (role.id=2b9a144dd3114f13a8fa1006f585fb12, scope.project.id=b94024e8a165476b8cb055d2fb4b8c48, user.id=ee0e6021a9b44d8d83996df691f23d8b) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:19:56.657637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:56.659046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506698b8-ea06-453c-be1b-a6f4ed92b904 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:56.664575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 428/856] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:19:56 2022] GET /identity/v3/role_assignments?user.id=ee0e6021a9b44d8d83996df691f23d8b&scope.project.id=b94024e8a165476b8cb055d2fb4b8c48&role.id=2b9a144dd3114f13a8fa1006f585fb12 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:58.791890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f4890-9d15-40cb-b1cf-e35ff5fd3ce1 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.792422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f4890-9d15-40cb-b1cf-e35ff5fd3ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.792541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f4890-9d15-40cb-b1cf-e35ff5fd3ce1 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.793837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 429/857] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:19:58.799114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:58.799515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:58.799676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:58.817210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:19:58.818428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:19:58.858428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:58.881825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 429/858] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:19:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:19:59.148415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f4890-9d15-40cb-b1cf-e35ff5fd3ce1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.241123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:19:59.248866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f4890-9d15-40cb-b1cf-e35ff5fd3ce1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.284354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe5e4c5-f42f-4d1a-ba09-018f1e3b5e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.284977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e4c5-f42f-4d1a-ba09-018f1e3b5e3e None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.285259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e4c5-f42f-4d1a-ba09-018f1e3b5e3e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.285633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe5e4c5-f42f-4d1a-ba09-018f1e3b5e3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.286661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5e4c5-f42f-4d1a-ba09-018f1e3b5e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.288092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe5e4c5-f42f-4d1a-ba09-018f1e3b5e3e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.289685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 430/859] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:19:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:19:59.295456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:19:59.390132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682397a6-7ce2-4cfc-9f9b-2ead6f1737f0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:19:59.423792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afab8978-05c8-402d-b4a1-b8baf81f5dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:19:59.424326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8978-05c8-402d-b4a1-b8baf81f5dd7 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:19:59.424603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8978-05c8-402d-b4a1-b8baf81f5dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:19:59.424873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afab8978-05c8-402d-b4a1-b8baf81f5dd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:19:59.425546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab8978-05c8-402d-b4a1-b8baf81f5dd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:19:59.426964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afab8978-05c8-402d-b4a1-b8baf81f5dd7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:19:59.453081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 430/860] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:19:59 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:01.618107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967d43-884c-40d8-89f1-0d6854a9d3b8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.618680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967d43-884c-40d8-89f1-0d6854a9d3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.618680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1967d43-884c-40d8-89f1-0d6854a9d3b8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.619916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 431/861] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:01.625823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:01.626210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:01.626378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:01.644681 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:01.645880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:01.667577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:01.711046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 431/862] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:01.978972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967d43-884c-40d8-89f1-0d6854a9d3b8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:02.078124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1967d43-884c-40d8-89f1-0d6854a9d3b8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.110560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ad6fb-da0d-4b0e-b32a-97c2b1a71cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.111166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ad6fb-da0d-4b0e-b32a-97c2b1a71cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.111445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ad6fb-da0d-4b0e-b32a-97c2b1a71cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.111715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ad6fb-da0d-4b0e-b32a-97c2b1a71cf5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.112810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ad6fb-da0d-4b0e-b32a-97c2b1a71cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:02.114262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ad6fb-da0d-4b0e-b32a-97c2b1a71cf5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:02.115789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 432/863] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:20:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:02.121742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:02.217426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51720138-be7f-4caa-8b69-90aa1337ada7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:02.250435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70bab0-a74f-46eb-bcbc-fdda5e639d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:02.250974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70bab0-a74f-46eb-bcbc-fdda5e639d21 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:02.251256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70bab0-a74f-46eb-bcbc-fdda5e639d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:02.251530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70bab0-a74f-46eb-bcbc-fdda5e639d21 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:02.252173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70bab0-a74f-46eb-bcbc-fdda5e639d21 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:02.253615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f70bab0-a74f-46eb-bcbc-fdda5e639d21 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:02.276022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 432/864] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:02 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:04.422143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5eae0-ff4e-4996-b195-f43c61dc6859 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.422690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5eae0-ff4e-4996-b195-f43c61dc6859 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.422690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d5eae0-ff4e-4996-b195-f43c61dc6859 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.424002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 433/865] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:04.429118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.429589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.429700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.447821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:04.448996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:04.458553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:04.515395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 433/866] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:04.784039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5eae0-ff4e-4996-b195-f43c61dc6859 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:04.814866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:04.883732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d5eae0-ff4e-4996-b195-f43c61dc6859 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:04.916086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:04.916687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:04.916962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:04.917229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:04.927871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:04.932569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:04.933707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:20:04.934563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 434/867] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:04.940455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.037257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bc71c-8948-4547-82ac-17e052c4413e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.068897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.069512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.069796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.070065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.070717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:05.071017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.072371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.080769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 434/868] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:05.086568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.175398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.184139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43ec24-b43e-4cd8-b72b-3ca7d3c79b6a None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.221709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.222321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.222596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.222865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] PATH_INFO: `/v3/users/2da3256c2cc54bf8b90c76a35d579796` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.236893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] RBAC: Authorizing `identity:get_user(user_id=2da3256c2cc54bf8b90c76a35d579796)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.238369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.239965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 435/869] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/users/2da3256c2cc54bf8b90c76a35d579796 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:05.246027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.255204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:05.345013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3291bec-9e4e-41d9-98eb-5d976f05e14c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.377637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4b61f-60f2-44e2-a555-76da82c3d350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.378250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4b61f-60f2-44e2-a555-76da82c3d350 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.378528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4b61f-60f2-44e2-a555-76da82c3d350 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.378797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4b61f-60f2-44e2-a555-76da82c3d350 None admin] PATH_INFO: `/v3/projects/e4eb48c3d9e243daab8fd98b3e193f6c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.379904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4b61f-60f2-44e2-a555-76da82c3d350 None admin] RBAC: Authorizing `identity:get_project(project_id=e4eb48c3d9e243daab8fd98b3e193f6c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.381332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4b61f-60f2-44e2-a555-76da82c3d350 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.382949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 435/870] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:05 2022] GET /identity/v3/projects/e4eb48c3d9e243daab8fd98b3e193f6c => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:05.388997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:05.486699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15f824-aa8f-4527-9486-fd03ec54cde1 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:05.519417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d5869f-6172-49b1-8abf-38fe49f77ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:05.520044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5869f-6172-49b1-8abf-38fe49f77ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:05.520319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5869f-6172-49b1-8abf-38fe49f77ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:05.520584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5869f-6172-49b1-8abf-38fe49f77ba8 None admin] PATH_INFO: `/v3/projects/e4eb48c3d9e243daab8fd98b3e193f6c/users/2da3256c2cc54bf8b90c76a35d579796/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:05.522419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5869f-6172-49b1-8abf-38fe49f77ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=e4eb48c3d9e243daab8fd98b3e193f6c, user_id=2da3256c2cc54bf8b90c76a35d579796, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:05.523823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d5869f-6172-49b1-8abf-38fe49f77ba8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:05.536502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 436/871] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:05 2022] PUT /identity/v3/projects/e4eb48c3d9e243daab8fd98b3e193f6c/users/2da3256c2cc54bf8b90c76a35d579796/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:07.639590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049311d4-0677-4286-a7e3-1e7bb35051a8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.640086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049311d4-0677-4286-a7e3-1e7bb35051a8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.640086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049311d4-0677-4286-a7e3-1e7bb35051a8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.641380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 436/872] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:07.646763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:07.647160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:07.647308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:07.665786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:07.666967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:07.669792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:07.733741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 437/873] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:08.000957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049311d4-0677-4286-a7e3-1e7bb35051a8 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.074415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.100844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049311d4-0677-4286-a7e3-1e7bb35051a8 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.132707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.133345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.133631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.133902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.144561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:08.149263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:08.150410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:20:08.151664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 437/874] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:08.157118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.251171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da78c8fa-41b8-4264-a10e-c728414241ed None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.284193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.284793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.285071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.285372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.286029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:08.286329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.287693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.295580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 438/875] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:08.301396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.398680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a491b-8469-4ddd-937e-3c8fec29b94e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.432756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.433407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.433690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.433965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] PATH_INFO: `/v3/users/2da3256c2cc54bf8b90c76a35d579796` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.435077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] RBAC: Authorizing `identity:get_user(user_id=2da3256c2cc54bf8b90c76a35d579796)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.437191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.438782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 438/876] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/users/2da3256c2cc54bf8b90c76a35d579796 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:08.444523 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.447370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.541898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ad9f1b-4835-4ed8-87e8-06bb96fa1faa None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.574499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.575085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.575362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.575629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.640956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.642405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.644117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.736120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 439/877] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:20:08.741999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.744848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:08.840439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cf7ffe-9c65-4bec-9cd6-5927924c2c2e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.873065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1356b200-3206-48c7-beaf-27cec588f9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:08.873850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b200-3206-48c7-beaf-27cec588f9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:08.874127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b200-3206-48c7-beaf-27cec588f9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:08.874398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1356b200-3206-48c7-beaf-27cec588f9d2 None admin] PATH_INFO: `/v3/projects/e4eb48c3d9e243daab8fd98b3e193f6c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:08.875512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356b200-3206-48c7-beaf-27cec588f9d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e4eb48c3d9e243daab8fd98b3e193f6c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:08.876965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1356b200-3206-48c7-beaf-27cec588f9d2 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:08.878585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 439/878] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/projects/e4eb48c3d9e243daab8fd98b3e193f6c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:08.884537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:08.980567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c5546a-8176-4c1d-8be6-96f43e5ceec4 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:08.982096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:09.015269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:09.015857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:09.016137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:09.016411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:09.017129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=e4eb48c3d9e243daab8fd98b3e193f6c, user.id=2da3256c2cc54bf8b90c76a35d579796) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:09.017469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:09.018922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f379651c-61d2-46bc-858f-41ec2537bb46 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:09.024428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 440/879] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:08 2022] GET /identity/v3/role_assignments?user.id=2da3256c2cc54bf8b90c76a35d579796&scope.project.id=e4eb48c3d9e243daab8fd98b3e193f6c&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:11.168319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb6b2a-d5fa-4e73-92eb-9f21e2aafe54 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.168874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb6b2a-d5fa-4e73-92eb-9f21e2aafe54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.168874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcb6b2a-d5fa-4e73-92eb-9f21e2aafe54 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.170971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 440/880] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:11.176111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a08a50-93a7-42a2-8622-ce1852b2b462 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.176519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a08a50-93a7-42a2-8622-ce1852b2b462 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.176654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a08a50-93a7-42a2-8622-ce1852b2b462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.194256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-21a08a50-93a7-42a2-8622-ce1852b2b462 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:11.195438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21a08a50-93a7-42a2-8622-ce1852b2b462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:11.264562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 441/881] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:11.537257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb6b2a-d5fa-4e73-92eb-9f21e2aafe54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:11.604326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:11.637920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcb6b2a-d5fa-4e73-92eb-9f21e2aafe54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:11.671165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518f5ff8-bdc8-4f87-bb7b-951d0087628a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:11.671746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f5ff8-bdc8-4f87-bb7b-951d0087628a None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:11.672021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f5ff8-bdc8-4f87-bb7b-951d0087628a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:11.672285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518f5ff8-bdc8-4f87-bb7b-951d0087628a None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:11.672945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f5ff8-bdc8-4f87-bb7b-951d0087628a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:11.674325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518f5ff8-bdc8-4f87-bb7b-951d0087628a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:11.689937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 441/882] 10.222.0.17 () {60 vars in 1138 bytes} [Wed Oct 12 12:20:11 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:13.820367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9540c8-a2aa-4d3f-be88-ccbf44b2056a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:13.820918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9540c8-a2aa-4d3f-be88-ccbf44b2056a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:13.820918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9540c8-a2aa-4d3f-be88-ccbf44b2056a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:13.822255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 442/883] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:13.828348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:13.828729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:13.828894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:13.847158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:13.848337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:13.912535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 442/884] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:14.183248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9540c8-a2aa-4d3f-be88-ccbf44b2056a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.275046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:14.282940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9540c8-a2aa-4d3f-be88-ccbf44b2056a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.316192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f47d1c-be00-4bbb-87d0-5f7605100ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.316808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f47d1c-be00-4bbb-87d0-5f7605100ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.317086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f47d1c-be00-4bbb-87d0-5f7605100ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.317384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f47d1c-be00-4bbb-87d0-5f7605100ba8 None admin] PATH_INFO: `/v3/domains/b84630d239e645d38996d8b66c5c195e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.327459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f47d1c-be00-4bbb-87d0-5f7605100ba8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b84630d239e645d38996d8b66c5c195e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.328893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f47d1c-be00-4bbb-87d0-5f7605100ba8 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.330501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 443/885] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Oct 12 12:20:14 2022] GET /identity/v3/domains/b84630d239e645d38996d8b66c5c195e => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:14.336242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:14.392927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:14.433331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715de892-8f1e-4efa-b549-8a0ab85dda96 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:14.465155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c114768-e501-41c0-9ffc-c602ebd8d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:14.465780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114768-e501-41c0-9ffc-c602ebd8d9eb None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:14.466058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114768-e501-41c0-9ffc-c602ebd8d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:14.466329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c114768-e501-41c0-9ffc-c602ebd8d9eb None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:14.467580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c114768-e501-41c0-9ffc-c602ebd8d9eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:14.469034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c114768-e501-41c0-9ffc-c602ebd8d9eb None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:14.494730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 443/886] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:20:14 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:16.657661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563006b-058e-46bd-a212-651ad4e97533 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.658200 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563006b-058e-46bd-a212-651ad4e97533 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.658200 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563006b-058e-46bd-a212-651ad4e97533 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.659472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 444/887] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:16.664692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:16.665109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:16.665247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:16.682952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:16.684129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:16.748866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 444/888] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:17.023276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563006b-058e-46bd-a212-651ad4e97533 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.123380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563006b-058e-46bd-a212-651ad4e97533 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.156468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1713938-65f2-4b18-b7eb-f9f83952d230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.157086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1713938-65f2-4b18-b7eb-f9f83952d230 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.157393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1713938-65f2-4b18-b7eb-f9f83952d230 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.157671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1713938-65f2-4b18-b7eb-f9f83952d230 None admin] PATH_INFO: `/v3/domains/b84630d239e645d38996d8b66c5c195e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.158777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1713938-65f2-4b18-b7eb-f9f83952d230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b84630d239e645d38996d8b66c5c195e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.160203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1713938-65f2-4b18-b7eb-f9f83952d230 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.161775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 445/889] 10.222.0.17 () {56 vars in 1183 bytes} [Wed Oct 12 12:20:17 2022] GET /identity/v3/domains/b84630d239e645d38996d8b66c5c195e => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:17.167435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:17.208510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:17.264453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bca13ce-4ccd-4f4a-b37f-e8b2d1981bc9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:17.298671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b24e44-4f79-4a27-b946-368a0c5ea3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:17.299202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24e44-4f79-4a27-b946-368a0c5ea3df None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:17.299477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24e44-4f79-4a27-b946-368a0c5ea3df None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:17.299745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b24e44-4f79-4a27-b946-368a0c5ea3df None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:17.300380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b24e44-4f79-4a27-b946-368a0c5ea3df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:17.302183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b24e44-4f79-4a27-b946-368a0c5ea3df None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:17.325118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 445/890] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:20:17 2022] POST /identity/v3/users => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:20:19.482168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5732-c719-49e3-b58d-f2658d6b5a2e None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.482716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5732-c719-49e3-b58d-f2658d6b5a2e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.482990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b5732-c719-49e3-b58d-f2658d6b5a2e None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.484325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 446/891] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:19.489397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.489829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.489939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.508317 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:19.509513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:19.573200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 446/892] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:19.838366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5732-c719-49e3-b58d-f2658d6b5a2e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:19.863339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:19.938329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b5732-c719-49e3-b58d-f2658d6b5a2e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:19.972211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:19.972823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:19.973103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:19.973399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:19.983759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:19.988504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:19.989699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:20:19.990846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 447/893] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:19.995703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.023058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.093136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab981f-ea43-4e2a-a8d3-0761aad6b784 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.127120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.127724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.127980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.128244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.128881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:20.129173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.130543 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.138145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 447/894] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:20.144006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.226309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.242161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f54749-2174-42bc-9b8a-335573afb218 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.274560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.275182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.275458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.275727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] PATH_INFO: `/v3/users/352151ae958f4c5ca4066967cc5b9523` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.289089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] RBAC: Authorizing `identity:get_user(user_id=352151ae958f4c5ca4066967cc5b9523)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.290564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.292150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 448/895] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/users/352151ae958f4c5ca4066967cc5b9523 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:20.298640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.385531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:20.395547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0946ab44-1048-44ae-b66a-8bb4f8dd5c01 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.428407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01402879-63e0-4b18-8a40-972fae1b3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.429013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01402879-63e0-4b18-8a40-972fae1b3b63 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.429294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01402879-63e0-4b18-8a40-972fae1b3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.429599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01402879-63e0-4b18-8a40-972fae1b3b63 None admin] PATH_INFO: `/v3/projects/30a33d7ce3be4f30b079332ad02c225b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.430704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01402879-63e0-4b18-8a40-972fae1b3b63 None admin] RBAC: Authorizing `identity:get_project(project_id=30a33d7ce3be4f30b079332ad02c225b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.432124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01402879-63e0-4b18-8a40-972fae1b3b63 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.433781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 448/896] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:20 2022] GET /identity/v3/projects/30a33d7ce3be4f30b079332ad02c225b => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:20.440046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:20.536486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8077242-dcb1-4fd5-9f85-4e6c744b04d2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:20.568732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc33e0-57b3-4b0c-9284-4fb42309735e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:20.569393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc33e0-57b3-4b0c-9284-4fb42309735e None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:20.569673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc33e0-57b3-4b0c-9284-4fb42309735e None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:20.569940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc33e0-57b3-4b0c-9284-4fb42309735e None admin] PATH_INFO: `/v3/projects/30a33d7ce3be4f30b079332ad02c225b/users/352151ae958f4c5ca4066967cc5b9523/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:20.571808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc33e0-57b3-4b0c-9284-4fb42309735e None admin] RBAC: Authorizing `identity:create_grant(project_id=30a33d7ce3be4f30b079332ad02c225b, user_id=352151ae958f4c5ca4066967cc5b9523, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:20.573228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc33e0-57b3-4b0c-9284-4fb42309735e None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:20.585909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 449/897] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:20:20 2022] PUT /identity/v3/projects/30a33d7ce3be4f30b079332ad02c225b/users/352151ae958f4c5ca4066967cc5b9523/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:20:22.784291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa920b1-d0d0-485a-9a07-e3a83098fd86 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.784848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa920b1-d0d0-485a-9a07-e3a83098fd86 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.784848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa920b1-d0d0-485a-9a07-e3a83098fd86 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.786180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 449/898] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:20:22.791467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:22.791896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:22.792000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:22.798373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:22.811409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:20:22.812593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:20:22.877041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 450/899] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:20:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:20:23.179047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa920b1-d0d0-485a-9a07-e3a83098fd86 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:23.278758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa920b1-d0d0-485a-9a07-e3a83098fd86 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:23.311317 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:23.311936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:23.312214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:23.312487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:23.327305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:20:23.332519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:20:23.333624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:20:23.334466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 450/900] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:20:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:20:23.339620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:23.438464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a866284-ccb4-4418-93fd-b84929c638cd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:23.466946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:23.472259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:23.472864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:23.473145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:23.473445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:23.474107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:23.474410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:23.475778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:23.483902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 451/901] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:20:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:23.489787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:23.509114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:23.587071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd804fbb-dc75-451b-9e0b-86f950cde9b3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:23.619972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:23.620582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:23.620861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:23.621426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] PATH_INFO: `/v3/users/352151ae958f4c5ca4066967cc5b9523` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:23.622534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] RBAC: Authorizing `identity:get_user(user_id=352151ae958f4c5ca4066967cc5b9523)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:23.624645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:23.626216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 451/902] 10.222.0.17 () {56 vars in 1177 bytes} [Wed Oct 12 12:20:23 2022] GET /identity/v3/users/352151ae958f4c5ca4066967cc5b9523 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:23.631874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:23.726116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262cebc-16d1-4363-827f-6d8e5402dff5 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:23.759134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:23.759716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:23.759992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:23.760260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:23.817116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:23.824490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:23.826105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:23.918996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 452/903] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:20:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:20:23.925458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:24.008250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:24.023533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cf5bf-e838-4fe6-b032-a7f8d79cd5e7 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:24.056408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf6394d-a881-43a6-86a4-f3e2d0925a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:24.057022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6394d-a881-43a6-86a4-f3e2d0925a6c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:24.057300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6394d-a881-43a6-86a4-f3e2d0925a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:24.057596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf6394d-a881-43a6-86a4-f3e2d0925a6c None admin] PATH_INFO: `/v3/projects/30a33d7ce3be4f30b079332ad02c225b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:24.058699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6394d-a881-43a6-86a4-f3e2d0925a6c None admin] RBAC: Authorizing `identity:get_project(project_id=30a33d7ce3be4f30b079332ad02c225b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:24.060149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf6394d-a881-43a6-86a4-f3e2d0925a6c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:24.061764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 452/904] 10.222.0.17 () {56 vars in 1186 bytes} [Wed Oct 12 12:20:23 2022] GET /identity/v3/projects/30a33d7ce3be4f30b079332ad02c225b => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:24.067771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:20:24.156601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:24.163375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdf53f-d1d2-46d3-8b47-c0dacbe85828 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:20:24.196308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:20:24.196893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:20:24.197170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:20:24.197468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:20:24.198178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=30a33d7ce3be4f30b079332ad02c225b, user.id=352151ae958f4c5ca4066967cc5b9523) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:20:24.198479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:20:24.199888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a1a7b-0280-4c04-a67e-ee8f15803009 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:20:24.205949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 453/905] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:20:24 2022] GET /identity/v3/role_assignments?user.id=352151ae958f4c5ca4066967cc5b9523&scope.project.id=30a33d7ce3be4f30b079332ad02c225b&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:20:34.052381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=23 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:20:34.203403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=21 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:19.719717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b538f2-4359-4b4d-9161-a9bc9cb0fd66 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:19.720420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b538f2-4359-4b4d-9161-a9bc9cb0fd66 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:19.720420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b538f2-4359-4b4d-9161-a9bc9cb0fd66 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:19.721756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 453/906] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:19.727940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:19.728429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:19.728518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:19.751727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:19.752999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:19.844424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 454/907] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:20.136225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b538f2-4359-4b4d-9161-a9bc9cb0fd66 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:20.262898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b538f2-4359-4b4d-9161-a9bc9cb0fd66 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:20.304804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:20.305501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.305797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.306073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.322113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:20.323616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:20.357840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:20.360642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:20.363123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:20.363123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:20.363123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:20.366491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 454/908] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:20.372662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:20.557296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaaaa23-1a7d-4809-816a-35e0ee044e18 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:20.622669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:20.626511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:20.627678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.627963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.628425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.629561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:20.630418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:20.633340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ec9182-b724-487a-9194-c2f6b3f37ccc None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:20.654779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 455/909] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:20.661395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:20.724312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:20.776821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ce032-725b-4880-955c-c3bc542e5e32 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:20.816134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be0a67-c1c1-4519-8a56-0c9ad1629894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:20.817008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0a67-c1c1-4519-8a56-0c9ad1629894 None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:20.817453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0a67-c1c1-4519-8a56-0c9ad1629894 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:20.817876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be0a67-c1c1-4519-8a56-0c9ad1629894 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:20.818674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be0a67-c1c1-4519-8a56-0c9ad1629894 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:20.820371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be0a67-c1c1-4519-8a56-0c9ad1629894 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:20.853501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 455/910] 10.222.0.17 () {60 vars in 1132 bytes} [Wed Oct 12 12:21:20 2022] POST /identity/v3/users => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:21:23.226644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48d5b2-60c0-429b-86c3-658406d8a13d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.227261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48d5b2-60c0-429b-86c3-658406d8a13d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.227426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e48d5b2-60c0-429b-86c3-658406d8a13d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.228749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 456/911] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:23.235954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.236489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.236489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.260013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:23.261240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:23.348094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 456/912] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:23.660520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48d5b2-60c0-429b-86c3-658406d8a13d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:23.810263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e48d5b2-60c0-429b-86c3-658406d8a13d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:23.855391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:23.856057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:23.856389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:23.856661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:23.869252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:23.875645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:23.876803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:21:23.877803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:23.883283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 457/913] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:23.886674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:23.934049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.016097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce51f8d-11dc-4629-88dd-bb0b7d82cc7f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.053907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.054602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.054877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.055150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.055813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:24.056117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.057531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.066941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 457/914] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:21:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:24.073502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.131001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.185213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a003e6-13b3-4290-96af-73e7e48ab800 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.224888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.225901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.226323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.226511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.239222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.241492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:24.258594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:24.259591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:24.260477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:24.260477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:24.260477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:24.262620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 458/915] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:24.269094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.382488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.385806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef52c84-b847-482f-a701-ab2ef1bb51d9 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.423318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.423986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.424304 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.424617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.425295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:24.425629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.427003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.440618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 458/916] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:24.446224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.566672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06030ef9-8f40-4ba8-807e-543234c85771 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.568694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:24.605645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.606292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.608919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.608919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.615624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.617077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:24.629488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:24.630521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:24.631493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 459/917] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:24.638001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.778605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e73199-49fc-4c10-acff-f763e2172abe None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:24.825796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:24.827040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:24.827040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:24.827218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:24.827939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:24.828782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:24.830544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:24.851464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 459/918] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:24.857233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:24.975500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236b482-678a-41df-8d5d-d3439b72f5f6 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.019588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.020641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.021170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.021682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.033759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.035467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:25.052111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:25.053184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:25.054220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:25.054220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:25.054220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:25.061117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 460/919] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:25.063138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.086057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.185059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c8ac5e-0562-4bc9-aa59-c96854845f2e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.228367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.229471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.229841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.230178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.230894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:25.231257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.232870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.248949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 460/920] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:25.255586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.276319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.379129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4a8a5-041b-4bd3-b8cc-de5b5a0fa9be None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.424395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.425288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.425600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.425872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.432763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.434342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:25.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:25.449025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:25.450332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:25.454601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 461/921] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:25.455851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.546277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.580704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bf3a5-3b72-492a-b800-245317a2c63a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.625912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.628328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.628328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.628328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.628609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:25.629417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.630216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc24c490-7fe1-4c2b-8cb9-19fe7b6f7611 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.648438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 461/922] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:25.654616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:25.743378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:25.777165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acf90-dfa9-4d5f-a4a6-d96fbd228e64 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:25.831107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16e9e9-e5fa-473c-9d10-6b1947070983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:25.831805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16e9e9-e5fa-473c-9d10-6b1947070983 None admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:25.832094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16e9e9-e5fa-473c-9d10-6b1947070983 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:25.832372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16e9e9-e5fa-473c-9d10-6b1947070983 None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/8740a4be0475453fb422383b9b335d86/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:25.853067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16e9e9-e5fa-473c-9d10-6b1947070983 None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=8740a4be0475453fb422383b9b335d86, role_id=3c5ebb59373d4fd180f8e6dbb0b8bb41)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:25.854822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f16e9e9-e5fa-473c-9d10-6b1947070983 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:25.870772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 462/923] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:25 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/8740a4be0475453fb422383b9b335d86/roles/3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:21:28.056754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69322c2d-15d4-4759-87fd-f57b7e493501 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.057292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69322c2d-15d4-4759-87fd-f57b7e493501 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.057292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69322c2d-15d4-4759-87fd-f57b7e493501 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.058657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 462/924] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:28.064096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.064531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.064648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.085533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:28.086921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:28.157430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 463/925] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:28.427902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69322c2d-15d4-4759-87fd-f57b7e493501 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:28.538407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69322c2d-15d4-4759-87fd-f57b7e493501 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:28.575491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:28.576288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.576641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.576994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.585736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:28.591027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:28.597852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:28.601435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:28.602642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 463/926] 10.222.0.17 () {56 vars in 1102 bytes} [Wed Oct 12 12:21:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:28.606888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:28.669733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:28.723323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1434c8ae-74d4-4930-aee7-a230193a0a7a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:28.760434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:28.761039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.761346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.761645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.762286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] RBAC: Adding query filter params (name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:28.762589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:28.763964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:28.772850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 464/927] 10.222.0.17 () {56 vars in 1103 bytes} [Wed Oct 12 12:21:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:28.778965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:28.893458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8432804-29b4-4e94-88d7-dc5b754f1761 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:28.930702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:28.931425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:28.931729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:28.932100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:28.944813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:28.946342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:28.962458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:28.963601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:28.969743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:28.969743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:28.969743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:28.969743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 464/928] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:28.970018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.089788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b822937d-ee97-4628-a9c4-e5d9c8f178af None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.107928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.128655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.129340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.129687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.129959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.130589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:29.130891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.132257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.146590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 465/929] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:29.153324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.209125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.263170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ba1df7-8c88-492a-a4e6-9f5d89009b7e None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.298608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.299297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.299631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.299941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.308956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.310456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:29.323193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:29.324526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:29.325844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:29.330150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 465/930] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:29.331368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.441352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a951b9b-e79b-40a9-a4d1-74aaeeb3a849 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.478094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.478764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.479088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.479381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.480123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:29.480493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.482061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.500933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 466/931] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:29.507661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.624185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eccfb-a94b-4613-bb8a-c3ed48a705de None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:29.638428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.666103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:29.666714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:29.666990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:29.667258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:29.749003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:29.751466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:29.856834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 466/932] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 12 12:21:29.863172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:29.900092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:29.970482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2612fc-0160-4f96-8963-b096f5711b72 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.007136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.007695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.007978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.008247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.019620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.021073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:30.035988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:30.042119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:30.043348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:30.043348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:30.043348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:30.043348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 467/933] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:30.043635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.056411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.160770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706fa442-0ac1-4d8c-8b64-91734d3cfc83 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.200704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.201419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.201772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.202147 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.202897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.203276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.204939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.217257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 467/934] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:30.224164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.325399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.342438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8292c9-bf28-46e5-bdc1-bab7b8f5d142 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.380069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.380869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.381256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.381584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.388724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.390187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:30.401851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:30.403000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:30.403922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 468/935] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:30.410168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.483036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:30.516311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760dd3c1-db61-4c09-9d53-0b6742474cb4 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.551839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.552445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.552727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.553000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.553715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.554022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.555448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65177e-9c28-402f-b9e8-b5749b2d68a1 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.568238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 468/936] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:30.574430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:30.682683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb38b8bf-9867-47e4-b7a9-60fcebaa6574 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:30.720038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:30.720791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:30.721074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:30.721451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:30.722429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] RBAC: Adding query filter params (role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=8740a4be0475453fb422383b9b335d86) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:30.722794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:30.724858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c20f9-a0ae-4476-a9d4-b8e28f194fb0 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:30.731693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 469/937] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:30 2022] GET /identity/v3/role_assignments?user.id=8740a4be0475453fb422383b9b335d86&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=3c5ebb59373d4fd180f8e6dbb0b8bb41 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:32.962938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e9fd5-6efa-4b97-bbba-4e7e6ee29947 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:32.963582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e9fd5-6efa-4b97-bbba-4e7e6ee29947 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:32.963582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210e9fd5-6efa-4b97-bbba-4e7e6ee29947 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:32.964886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 469/938] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:21:32.971184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:32.971627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:32.971725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:32.992593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:32.993981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:32.997506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:33.070389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 470/939] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:33.351439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e9fd5-6efa-4b97-bbba-4e7e6ee29947 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:33.469649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210e9fd5-6efa-4b97-bbba-4e7e6ee29947 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:33.500875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:33.507077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:33.507735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.508017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.508284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.519804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:33.524702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:33.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:33.527102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 470/940] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:21:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:33.532535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:33.648639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792a363-82d3-4915-a5d4-75b0cba225e2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:33.685723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:33.686310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.686630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.686962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.688300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:33.688586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:33.690102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:33.699148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 471/941] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:33.705841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:33.827839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af27832-6b01-4800-bce3-d4af7dff0121 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:33.867163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:33.867825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:33.868108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:33.868382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:33.879394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:33.881005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:33.897966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:33.899056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:33.900255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:33.900255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:33.900255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:33.900255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 471/942] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:33.905232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:33.908908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:34.017520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39668988-dabf-4017-8a6d-4dab3f15d5b2 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:34.058177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:34.058776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:34.059049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:34.059311 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:34.059938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:34.060235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:34.061632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:34.073327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 472/943] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:34.079525 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:34.124156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:34.185524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155a8cd-0ce6-4c73-abdb-ceeab05e018a None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:34.223506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:34.224413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:34.224795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:34.225077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:34.234707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:34.236497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:34.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:34.257536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:34.258745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 472/944] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:34.260528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:34.330626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:34.371198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22bcab6-189b-4974-8069-d5d51ba5567d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:34.408931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:34.409872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:34.410240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:34.410635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:34.411610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:34.412070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:34.414120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:34.433710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 473/945] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Oct 12 12:21:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:34.439858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:34.551316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79af79f-28b2-4dc9-8814-c9f307b5aa00 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:34.553110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:34.587388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:34.587998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:34.588276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:34.588547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:34.601180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:34.602783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:34.621682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:34.623078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:34.623957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:34.623957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:34.623957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:34.623957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 473/946] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:34.629439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:34.733848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c9a84-86e3-441b-aa80-0e6cf3996eb9 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:34.756094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:34.773004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:34.773652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:34.773936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:34.774208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:34.774855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:34.775158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:34.776666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:34.792430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 474/947] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:34.798283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:34.908965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767264e-4f04-4cf2-b5d8-4b8e8648ade6 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:34.945477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:34.946125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:34.946389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:34.946662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:34.954080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:34.955568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:34.968172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:34.970773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:34.971816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 474/948] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:34.976274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:35.097882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3bfb4d-9ec1-4c7c-9a6b-5946b4679759 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:35.134759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:35.135364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:35.135643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:35.135915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:35.136602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:35.136908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:35.138381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62737085-c04a-445c-9bd3-e0d38d07e9d1 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:35.152981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 475/949] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:35.159607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:35.178606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:35.278500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c9483-ec18-4f4d-9ff6-c13fc802f219 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:35.318405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4176e73-2997-4672-8e20-4c96de80221f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:35.319233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4176e73-2997-4672-8e20-4c96de80221f None admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:35.319641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4176e73-2997-4672-8e20-4c96de80221f None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:35.320105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4176e73-2997-4672-8e20-4c96de80221f None admin] PATH_INFO: `/v3/projects/4449657cec624fe4912405ed1757eff1/users/8740a4be0475453fb422383b9b335d86/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:35.322152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4176e73-2997-4672-8e20-4c96de80221f None admin] RBAC: Authorizing `identity:create_grant(project_id=4449657cec624fe4912405ed1757eff1, user_id=8740a4be0475453fb422383b9b335d86, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:35.323855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4176e73-2997-4672-8e20-4c96de80221f None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:35.338768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 475/950] 10.222.0.17 () {58 vars in 1439 bytes} [Wed Oct 12 12:21:35 2022] PUT /identity/v3/projects/4449657cec624fe4912405ed1757eff1/users/8740a4be0475453fb422383b9b335d86/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:21:37.620301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fe0d2-3447-4b0b-895e-60003470fa7d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:37.620833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fe0d2-3447-4b0b-895e-60003470fa7d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:37.620833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628fe0d2-3447-4b0b-895e-60003470fa7d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:37.622527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 476/951] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:37.628896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:37.629405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:37.629538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:37.648223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:37.649520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:37.723398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 476/952] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 12 12:21:38.015356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fe0d2-3447-4b0b-895e-60003470fa7d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:38.063570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:38.129444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628fe0d2-3447-4b0b-895e-60003470fa7d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:38.167210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:38.167926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.168264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.168574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.179694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 12 12:21:38.185070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:38.186330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 12 12:21:38.187372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:38.191757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 477/953] 10.222.0.17 () {56 vars in 1096 bytes} [Wed Oct 12 12:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:38.192710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:38.195817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:38.303752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221645b9-f22d-4c31-b230-e3808c5936b5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:38.339846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:38.340505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.340804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.341095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.341833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:38.342140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:38.344124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:38.352953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 477/954] 10.222.0.17 () {56 vars in 1099 bytes} [Wed Oct 12 12:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:38.360012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:38.466189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:38.471010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbbdaf9-b1d2-4bac-87d4-6d7d71f61a5d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:38.505120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:38.505858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.506133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.506406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.518497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:38.519990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:38.540209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:38.547110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:38.548198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:38.548198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:38.548198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:38.548198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 478/955] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:38.548437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:38.621695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:38.660698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd161bd8-f696-41f9-bc3d-a270738cb34b None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:38.697197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:38.697845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.698120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.698388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.699014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:38.699313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:38.700668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:38.715670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 478/956] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:38.722200 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:38.827188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c695f991-f864-44fe-be45-02dd50920c65 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:38.864603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:38.865227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:38.865537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:38.865812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:38.874193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:38.876175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 12 12:21:38.888241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:38.889442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 12 12:21:38.890422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:38.893494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 479/957] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 12 12:21:38.896171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.008055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085b8fa0-5b05-4f4f-a703-36f34059fe4c None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.037764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.046824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.047468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.047749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.048026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.048751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:39.049057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.050517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.068316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 479/958] 10.222.0.17 () {56 vars in 1143 bytes} [Wed Oct 12 12:21:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:39.074449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.091159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.183244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a5ae-db41-4847-a62d-09408acf13a3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.221356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.221974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.222255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.222526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.299362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.301009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.317118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.408759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 480/959] 10.222.0.17 () {58 vars in 1282 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 12 12:21:39.414703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.520380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245f57f-16e2-482b-b01b-81f6d2d481e3 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.556061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.556693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.556967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.557231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.567999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.569494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 12 12:21:39.585761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:39.586884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 12 12:21:39.588097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 12 12:21:39.588097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 12 12:21:39.588097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:39.588097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 480/960] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:39.592900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.695447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36fd435-d2da-4ba5-98ad-d1b1b1b23a54 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.709555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.730408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.731001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.731302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.731567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.732196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:39.732492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.734238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.745719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 481/961] 10.222.0.17 () {56 vars in 1109 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:39.752300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:39.788618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:39.855348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e9f839-45d0-48f1-9baa-fc8217b62972 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:39.894845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:39.897848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:39.897848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:39.897848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:39.902902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:39.904331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 12 12:21:39.915728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:39.916703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: ERROR keystone.server.flask.application Oct 12 12:21:39.917776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 481/962] 10.222.0.17 () {56 vars in 1111 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:39.922872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:40.037078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59a4535-713d-4073-84ce-764746c81259 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:40.070031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:40.070567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:40.070844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:40.071114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:40.071794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:40.072096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:40.073543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c38403-e408-4990-b82f-5c2064472cbe None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:40.084965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 482/963] 10.222.0.17 () {56 vars in 1148 bytes} [Wed Oct 12 12:21:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:40.091148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:40.181878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:40.188867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bd5238-c788-4f4c-b1f6-c233c3fd8092 None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:40.221241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:40.221854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:40.222132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:40.222403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:40.223114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] RBAC: Adding query filter params (role.id=2e857246137f4d06b40cdabedf42881f, scope.project.id=4449657cec624fe4912405ed1757eff1, user.id=8740a4be0475453fb422383b9b335d86) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:40.223421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:40.224844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e44b7-7d91-45cb-882b-4b9846632fea None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:40.230448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 482/964] 10.222.0.17 () {56 vars in 1374 bytes} [Wed Oct 12 12:21:40 2022] GET /identity/v3/role_assignments?user.id=8740a4be0475453fb422383b9b335d86&scope.project.id=4449657cec624fe4912405ed1757eff1&role.id=2e857246137f4d06b40cdabedf42881f => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:42.374151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455361f-579d-4a9f-9b00-b52af2bee7df None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.374756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455361f-579d-4a9f-9b00-b52af2bee7df None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.374756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b455361f-579d-4a9f-9b00-b52af2bee7df None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.376046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 483/965] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:42.381142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.381563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.381746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.401192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:42.402423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:42.465818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 483/966] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:42.730211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455361f-579d-4a9f-9b00-b52af2bee7df None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:42.760647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:42.829640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b455361f-579d-4a9f-9b00-b52af2bee7df None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:42.861683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:42.862289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:42.862566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:42.862837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:42.863404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:42.864766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:42.873566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:42.875356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:42.876633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 484/967] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:21:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:42.880976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:42.968521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:42.981449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f220e4-a78d-4509-8f54-f330368a4ec2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:43.025678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:43.026642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:43.027022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:43.027302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:43.027917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:43.028287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:43.029813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb76cce-33e1-400b-ac40-d878f0db95d5 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:43.034976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 484/968] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Oct 12 12:21:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:43.040882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:43.134926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:43.141389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f058c-1713-4743-8fbe-deeb10897516 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:43.173703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:43.174297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:43.174575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:43.174846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] PATH_INFO: `/v3/services` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:43.175463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:43.175760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:43.177134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f87b7-bbca-47ce-a9c4-d23d696a1004 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:43.181510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 485/969] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Oct 12 12:21:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:21:45.310413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3929e6-0b7f-4b92-ba37-235d4d9d43c6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.310914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3929e6-0b7f-4b92-ba37-235d4d9d43c6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.311024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3929e6-0b7f-4b92-ba37-235d4d9d43c6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.312297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 485/970] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:45.317625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aeb7b4-405b-42a3-8b1a-e22fb37d02c7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.318069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aeb7b4-405b-42a3-8b1a-e22fb37d02c7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.318170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aeb7b4-405b-42a3-8b1a-e22fb37d02c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.337197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-74aeb7b4-405b-42a3-8b1a-e22fb37d02c7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:45.338419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-74aeb7b4-405b-42a3-8b1a-e22fb37d02c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:45.403426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 486/971] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:45.670352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3929e6-0b7f-4b92-ba37-235d4d9d43c6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:45.768940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3929e6-0b7f-4b92-ba37-235d4d9d43c6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:45.801612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aae140-ec73-4547-bc25-bd2bcfd7321e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:45.802199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae140-ec73-4547-bc25-bd2bcfd7321e None admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:45.802472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae140-ec73-4547-bc25-bd2bcfd7321e None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:45.802739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aae140-ec73-4547-bc25-bd2bcfd7321e None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:45.803376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aae140-ec73-4547-bc25-bd2bcfd7321e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:45.804721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40aae140-ec73-4547-bc25-bd2bcfd7321e None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:45.819357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 486/972] 10.222.0.17 () {60 vars in 1141 bytes} [Wed Oct 12 12:21:45 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:21:47.961741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb231-37b7-4a23-8f3c-a2aa32e07841 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:47.962298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb231-37b7-4a23-8f3c-a2aa32e07841 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:47.962298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bdb231-37b7-4a23-8f3c-a2aa32e07841 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:47.963538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 487/973] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:47.970232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:47.970699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:47.970810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:47.989782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:47.991001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:48.001743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:48.069691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 487/974] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:47 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:48.336702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdb231-37b7-4a23-8f3c-a2aa32e07841 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.425714 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:48.436649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bdb231-37b7-4a23-8f3c-a2aa32e07841 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.470943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.471566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.471846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.472117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.472685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.474148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:48.482825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:48.484012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:48.485093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 488/975] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:48.489891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.584599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced5e1a-9cfd-4d6f-8e06-6bdb3c3a2b49 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.616871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.617483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.617759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.618021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.618636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:48.618933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.620283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.625447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 488/976] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:48.631220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.728064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542dc4b5-c268-4812-a273-67abf38c48c3 None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.761102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.761722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.762004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.762280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.762968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] RBAC: Adding query filter params (interface=public, service_id=6d43a8f13f314520b977f0ed2137389c, region_id=RegionOne) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:48.763276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.764640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e663a06-af0e-47e5-b8b6-a81f67f0427a None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.769735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 489/977] 10.222.0.17 () {56 vars in 1251 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/endpoints?service_id=6d43a8f13f314520b977f0ed2137389c&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:48.775228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:48.778875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:48.873336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f345-6df5-4f16-bc5b-cfc64370d3cc None admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:48.905528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091c2780-a151-4c1a-9365-97707de03c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:48.906598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c2780-a151-4c1a-9365-97707de03c33 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:48.906874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c2780-a151-4c1a-9365-97707de03c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:48.907141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091c2780-a151-4c1a-9365-97707de03c33 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:48.907728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c2780-a151-4c1a-9365-97707de03c33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:48.909069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091c2780-a151-4c1a-9365-97707de03c33 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:48.915606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 489/978] 10.222.0.17 () {56 vars in 1087 bytes} [Wed Oct 12 12:21:48 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:21:51.029857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594e28a-1d65-4c11-97df-f979f823b55b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.030418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594e28a-1d65-4c11-97df-f979f823b55b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.030418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a594e28a-1d65-4c11-97df-f979f823b55b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.033233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 490/979] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:21:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:21:51.039244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.039657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.039789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.059242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:21:51.060416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:21:51.119301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:51.125632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 490/980] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:21:51 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 12 12:21:51.392383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594e28a-1d65-4c11-97df-f979f823b55b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:51.411702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:51.490329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a594e28a-1d65-4c11-97df-f979f823b55b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:51.522736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:51.523329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.523604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.523870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.524432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:51.525811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 12 12:21:51.534807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 12 12:21:51.535859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.get_or_create( Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application with Lock( Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self._enter() Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application return self.creator() Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application created_value = creator( Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: ERROR keystone.server.flask.application Oct 12 12:21:51.536734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 491/981] 10.222.0.17 () {56 vars in 1117 bytes} [Wed Oct 12 12:21:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 12 12:21:51.541900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:51.637371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4671d51-50a4-4060-bb76-8046c6e06d6a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:51.669922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:51.670448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.670721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.670990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] PATH_INFO: `/v3/services` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.671603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:21:51.672040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:51.673907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c168f-72c6-4b84-972b-198f8ae91403 None admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:51.678843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 491/982] 10.222.0.17 () {56 vars in 1116 bytes} [Wed Oct 12 12:21:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:21:51.684490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:21:51.766276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:21:51.784626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c5cfc-9f00-4aa6-9b30-2f8bbe2ef11d None admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:21:51.817660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1328dbce-0309-44b0-a0c9-c87c5588404d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:21:51.818246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328dbce-0309-44b0-a0c9-c87c5588404d None admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:21:51.818518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328dbce-0309-44b0-a0c9-c87c5588404d None admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:21:51.818784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1328dbce-0309-44b0-a0c9-c87c5588404d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:21:51.819442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328dbce-0309-44b0-a0c9-c87c5588404d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:21:51.820788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1328dbce-0309-44b0-a0c9-c87c5588404d None admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:21:51.840340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 492/983] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:21:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:22:01.676493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=69 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:01.836097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:55.399026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dd99d-1f22-4430-8070-35bbc56de098 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:55.400433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dd99d-1f22-4430-8070-35bbc56de098 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:55.401421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933dd99d-1f22-4430-8070-35bbc56de098 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:55.405205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 492/984] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:22:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:22:55.411204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d11dd-28f3-40b1-811b-a790fe872267 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:55.411634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d11dd-28f3-40b1-811b-a790fe872267 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:55.411800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d11dd-28f3-40b1-811b-a790fe872267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:55.539592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f57d11dd-28f3-40b1-811b-a790fe872267 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:22:55.540871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f57d11dd-28f3-40b1-811b-a790fe872267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:55.659581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 493/985] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:22:55.955560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa31961-b322-47b9-b301-efcbef24822b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:55.955861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa31961-b322-47b9-b301-efcbef24822b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:55.956273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa31961-b322-47b9-b301-efcbef24822b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:55.957910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 493/986] 10.222.0.17 () {54 vars in 964 bytes} [Wed Oct 12 12:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:22:55.964125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658399b-75a3-4932-b2ef-45f32ef6fcb3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:55.964411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658399b-75a3-4932-b2ef-45f32ef6fcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:55.964679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658399b-75a3-4932-b2ef-45f32ef6fcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:56.045343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1658399b-75a3-4932-b2ef-45f32ef6fcb3 None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:22:56.046561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1658399b-75a3-4932-b2ef-45f32ef6fcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:56.069216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:56.133548 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 494/987] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Oct 12 12:22:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:22:56.143485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa31961-b322-47b9-b301-efcbef24822b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:56.145281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1aa31961-b322-47b9-b301-efcbef24822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:56.273249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa31961-b322-47b9-b301-efcbef24822b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:56.365960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:56.366577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:56.366858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:56.367130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:56.368839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:56.430108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:56.493977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:56.495414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfed499-8ef9-4485-a10d-37040cfbc200 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:56.516689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 494/988] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:22:58.972639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b045a3-e3a2-4e52-b0b4-070701b0c881 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:58.973198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b045a3-e3a2-4e52-b0b4-070701b0c881 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:58.973198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b045a3-e3a2-4e52-b0b4-070701b0c881 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:58.974603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 495/989] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:22:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:22:58.980051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d7d26-d797-4dc4-af28-98fa11f0e3a3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:58.980452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d7d26-d797-4dc4-af28-98fa11f0e3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:58.980583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6d7d26-d797-4dc4-af28-98fa11f0e3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:59.062987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-db6d7d26-d797-4dc4-af28-98fa11f0e3a3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:22:59.064817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-db6d7d26-d797-4dc4-af28-98fa11f0e3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:59.161779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 495/990] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:22:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:22:59.441960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b045a3-e3a2-4e52-b0b4-070701b0c881 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:22:59.443499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d8b045a3-e3a2-4e52-b0b4-070701b0c881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:59.547321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b045a3-e3a2-4e52-b0b4-070701b0c881 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:22:59.618774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:22:59.620913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:22:59.621536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:22:59.622006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:22:59.622435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:22:59.624507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:22:59.734712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:22:59.736382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925ba5d8-f2ef-41e7-bbfd-ac01778c6ca1 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:22:59.757235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 496/991] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:22:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:02.254447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9d73d-3fa8-4dce-8ae2-fc6d9c3386f8 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:02.254994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9d73d-3fa8-4dce-8ae2-fc6d9c3386f8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:02.254994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9d73d-3fa8-4dce-8ae2-fc6d9c3386f8 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:02.256336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 496/992] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:02.261769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93c64-407b-46f3-a4c6-8aacf1ef8916 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:02.262118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93c64-407b-46f3-a4c6-8aacf1ef8916 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:02.262118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee93c64-407b-46f3-a4c6-8aacf1ef8916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:02.327126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0ee93c64-407b-46f3-a4c6-8aacf1ef8916 None None] MFA Rules not processed for user `c0edd4c51fd342338cf2652e2ee1df11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:02.328312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0ee93c64-407b-46f3-a4c6-8aacf1ef8916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:02.379486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:02.424319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 497/993] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:02.723964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82f8f1-4ed1-4a0a-977b-322ed1733e54 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:02.724242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82f8f1-4ed1-4a0a-977b-322ed1733e54 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:02.724513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f82f8f1-4ed1-4a0a-977b-322ed1733e54 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:02.725875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 497/994] 10.222.0.17 () {54 vars in 964 bytes} [Wed Oct 12 12:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:02.731755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d79fc-e6db-4f1d-9dc6-8557e242eb13 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:02.732032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d79fc-e6db-4f1d-9dc6-8557e242eb13 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:02.732296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3d79fc-e6db-4f1d-9dc6-8557e242eb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:02.808054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6e3d79fc-e6db-4f1d-9dc6-8557e242eb13 None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:02.809246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6e3d79fc-e6db-4f1d-9dc6-8557e242eb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:02.887561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 498/995] 10.222.0.17 () {58 vars in 1061 bytes} [Wed Oct 12 12:23:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:02.897266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82f8f1-4ed1-4a0a-977b-322ed1733e54 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:02.898840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f82f8f1-4ed1-4a0a-977b-322ed1733e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:02.908663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:03.001973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f82f8f1-4ed1-4a0a-977b-322ed1733e54 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:03.074152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:03.074745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:03.075025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:03.075295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:03.076878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:03.140005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:03.186577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:03.188017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7399f562-f9a6-4c54-8114-fc33eee291a8 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:03.207130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 498/996] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:06.255998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519409-72d7-4559-b47e-d8b366b3d6e5 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:06.256530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519409-72d7-4559-b47e-d8b366b3d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:06.256647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db519409-72d7-4559-b47e-d8b366b3d6e5 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:06.257960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 499/997] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:06.263418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c3f0a-8a89-4cd6-aa98-82b582612035 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:06.263794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c3f0a-8a89-4cd6-aa98-82b582612035 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:06.263990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c3f0a-8a89-4cd6-aa98-82b582612035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:06.328905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-905c3f0a-8a89-4cd6-aa98-82b582612035 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:06.330904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-905c3f0a-8a89-4cd6-aa98-82b582612035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:06.426463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 499/998] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:06.713240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519409-72d7-4559-b47e-d8b366b3d6e5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:06.714794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-db519409-72d7-4559-b47e-d8b366b3d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:06.765048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:06.817833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db519409-72d7-4559-b47e-d8b366b3d6e5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:06.890189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:06.890801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:06.891059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:06.891327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:06.893848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:06.988822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:07.003113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:07.004544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187a1abb-e569-4e59-b5e5-20a77e9a8ada service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:07.021921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 500/999] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:09.823472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91f9f-8456-451e-be69-d83deaf89770 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:09.824034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91f9f-8456-451e-be69-d83deaf89770 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:09.824034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b91f9f-8456-451e-be69-d83deaf89770 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:09.825359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 500/1000] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:09.830574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deee45-8a61-4045-8f72-e614bfa05619 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:09.830982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deee45-8a61-4045-8f72-e614bfa05619 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:09.831127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47deee45-8a61-4045-8f72-e614bfa05619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:09.901695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-47deee45-8a61-4045-8f72-e614bfa05619 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:09.902981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-47deee45-8a61-4045-8f72-e614bfa05619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:10.010431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 501/1001] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:10.304006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b91f9f-8456-451e-be69-d83deaf89770 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:10.305561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b9b91f9f-8456-451e-be69-d83deaf89770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:10.326145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:10.408777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b91f9f-8456-451e-be69-d83deaf89770 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:10.482100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:10.482698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:10.482974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:10.483239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:10.484806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:10.557581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:10.595841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:10.597474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc745094-6177-4a13-a714-e413871ddfab service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:10.616456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 501/1002] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:12.851848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba2f92-8c44-4626-99a0-37eec3effc52 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:12.852502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba2f92-8c44-4626-99a0-37eec3effc52 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:12.852502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba2f92-8c44-4626-99a0-37eec3effc52 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:12.853984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 502/1003] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:12.859097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e52c-26c7-4f84-89b2-0bb3434a0613 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:12.859498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e52c-26c7-4f84-89b2-0bb3434a0613 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:12.859646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b0e52c-26c7-4f84-89b2-0bb3434a0613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:12.927548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a2b0e52c-26c7-4f84-89b2-0bb3434a0613 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:12.928739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2b0e52c-26c7-4f84-89b2-0bb3434a0613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:13.035008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 502/1004] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:13.317274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba2f92-8c44-4626-99a0-37eec3effc52 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:13.318870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-55ba2f92-8c44-4626-99a0-37eec3effc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:13.380635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:13.425557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba2f92-8c44-4626-99a0-37eec3effc52 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:13.502566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:13.503174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:13.503451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:13.503729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:13.505352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:13.615596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:13.618248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:13.620058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5493a1f-166c-4acc-af4a-4b9fc64b6480 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:13.639131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 503/1005] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:15.844811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758fb8-32dc-4c2e-8195-5c0a5736e911 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:15.845379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758fb8-32dc-4c2e-8195-5c0a5736e911 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:15.845379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758fb8-32dc-4c2e-8195-5c0a5736e911 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:15.846713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 503/1006] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:15.852213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29af4-6df7-4bb4-8408-f980be5f8aca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:15.852616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29af4-6df7-4bb4-8408-f980be5f8aca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:15.852767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac29af4-6df7-4bb4-8408-f980be5f8aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:15.918114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bac29af4-6df7-4bb4-8408-f980be5f8aca None None] MFA Rules not processed for user `c0edd4c51fd342338cf2652e2ee1df11`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:15.919320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bac29af4-6df7-4bb4-8408-f980be5f8aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:16.017536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 504/1007] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:15 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:16.301612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78758fb8-32dc-4c2e-8195-5c0a5736e911 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:16.303157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-78758fb8-32dc-4c2e-8195-5c0a5736e911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:16.336010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:16.413258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78758fb8-32dc-4c2e-8195-5c0a5736e911 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:16.487925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:16.488534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:16.488814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:16.489085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:16.490717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:16.571054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:16.604959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:16.606455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc605a71-b14c-457f-acfa-7d1e97040c30 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:16.625654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 504/1008] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:19.016801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6ec7-34e3-4330-b5a5-c349696658bf None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:19.017344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6ec7-34e3-4330-b5a5-c349696658bf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:19.017600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16d6ec7-34e3-4330-b5a5-c349696658bf None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:19.019457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 505/1009] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:19.024886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56a14-43c6-483b-98c4-e7b56fb38341 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:19.025434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56a14-43c6-483b-98c4-e7b56fb38341 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:19.025541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d56a14-43c6-483b-98c4-e7b56fb38341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:19.090110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d3d56a14-43c6-483b-98c4-e7b56fb38341 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:19.091315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3d56a14-43c6-483b-98c4-e7b56fb38341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:19.185264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 505/1010] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:19.464762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6ec7-34e3-4330-b5a5-c349696658bf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:19.466323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e16d6ec7-34e3-4330-b5a5-c349696658bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:19.536672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:19.570621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16d6ec7-34e3-4330-b5a5-c349696658bf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:19.643596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:19.644189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:19.644465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:19.644732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:19.646334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:19.753147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:19.754603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbf5ff-6807-4e76-9c3f-e0c7366436fc service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:19.764317 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:19.773437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 506/1011] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:21.968377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2c54-8ecf-4608-8f98-a8b65c92e168 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:21.969077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2c54-8ecf-4608-8f98-a8b65c92e168 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:21.969077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f2c54-8ecf-4608-8f98-a8b65c92e168 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:21.970316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 506/1012] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:21.976185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2043a-4001-4ab2-a089-cb706d53ac3e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:21.976619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2043a-4001-4ab2-a089-cb706d53ac3e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:21.976736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba2043a-4001-4ab2-a089-cb706d53ac3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:22.042626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9ba2043a-4001-4ab2-a089-cb706d53ac3e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:22.043816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9ba2043a-4001-4ab2-a089-cb706d53ac3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:22.138541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 507/1013] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:22.425399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2c54-8ecf-4608-8f98-a8b65c92e168 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:22.426980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e48f2c54-8ecf-4608-8f98-a8b65c92e168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:22.468654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:22.534217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f2c54-8ecf-4608-8f98-a8b65c92e168 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:22.606901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:22.607504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:22.607780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:22.608047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:22.609677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:22.693693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:22.719372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:22.720873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0e858c-49df-4f0a-8793-67304a63d45a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:22.739588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 507/1014] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:25.342000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b9f69-fc70-4ad1-901f-c2d6d4031ba9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:25.342533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b9f69-fc70-4ad1-901f-c2d6d4031ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:25.342533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b9f69-fc70-4ad1-901f-c2d6d4031ba9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:25.343867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 508/1015] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:25.348978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a33eb3-9330-4901-94f8-101ed3e803a5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:25.349396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a33eb3-9330-4901-94f8-101ed3e803a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:25.349564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a33eb3-9330-4901-94f8-101ed3e803a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:25.415561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-88a33eb3-9330-4901-94f8-101ed3e803a5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:25.416769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-88a33eb3-9330-4901-94f8-101ed3e803a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:25.513596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 508/1016] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:25.799884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b9f69-fc70-4ad1-901f-c2d6d4031ba9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:25.801452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-556b9f69-fc70-4ad1-901f-c2d6d4031ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:25.877924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:25.904943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b9f69-fc70-4ad1-901f-c2d6d4031ba9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:25.977800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:25.978398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:25.978673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:25.978944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:25.980528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:26.088659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:26.090028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968d7487-4f41-45be-963a-1bca4cbe9cc1 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:26.108953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 509/1017] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:30.176495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ee20e-9474-45d8-b82b-c1e7d5b17b98 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:30.177044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ee20e-9474-45d8-b82b-c1e7d5b17b98 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:30.177044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ee20e-9474-45d8-b82b-c1e7d5b17b98 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:30.178541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 509/1018] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:30.183533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812e0c4-5c4d-4518-845d-ad91baa7c3a6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:30.183938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812e0c4-5c4d-4518-845d-ad91baa7c3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:30.184077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7812e0c4-5c4d-4518-845d-ad91baa7c3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:30.190182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:30.255215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7812e0c4-5c4d-4518-845d-ad91baa7c3a6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:30.256453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7812e0c4-5c4d-4518-845d-ad91baa7c3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:30.357357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 510/1019] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:30.642492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ee20e-9474-45d8-b82b-c1e7d5b17b98 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:30.644047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-708ee20e-9474-45d8-b82b-c1e7d5b17b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:30.693800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:30.750514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ee20e-9474-45d8-b82b-c1e7d5b17b98 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:30.825187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:30.825826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:30.826106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:30.826373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:30.827955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:30.925386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:30.941381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:30.942818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda94c4d-2aa5-4df7-9467-6a452740bb92 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:30.962119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 510/1020] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:33.944889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cd7b-9ea1-415f-a051-1cbdef9aafa9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.945438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cd7b-9ea1-415f-a051-1cbdef9aafa9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.945526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8cd7b-9ea1-415f-a051-1cbdef9aafa9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:33.946842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 511/1021] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:33.952389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110013-0fc0-4155-8d00-5b53df5cbb0f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:33.952796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110013-0fc0-4155-8d00-5b53df5cbb0f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:33.952941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f110013-0fc0-4155-8d00-5b53df5cbb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:34.024339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0f110013-0fc0-4155-8d00-5b53df5cbb0f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:34.025621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f110013-0fc0-4155-8d00-5b53df5cbb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:34.124768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 511/1022] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:34.409344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8cd7b-9ea1-415f-a051-1cbdef9aafa9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:34.410913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-59e8cd7b-9ea1-415f-a051-1cbdef9aafa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:34.498693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:34.517514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8cd7b-9ea1-415f-a051-1cbdef9aafa9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:34.593776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:34.594375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:34.594651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:34.594915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:34.596537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:34.726493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:34.727961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d5c0b-6b3c-4346-9f68-afb7cfc4e5d4 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:34.747195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 512/1023] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:39.121003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099247db-6355-459c-8ed4-9b9437ea0454 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:39.121519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099247db-6355-459c-8ed4-9b9437ea0454 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:39.121654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099247db-6355-459c-8ed4-9b9437ea0454 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:39.123003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 512/1024] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:39.130436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b23d-7d27-4c21-97fb-8276faf6e290 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:39.130871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b23d-7d27-4c21-97fb-8276faf6e290 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:39.130871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0894b23d-7d27-4c21-97fb-8276faf6e290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:39.148293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:39.209614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0894b23d-7d27-4c21-97fb-8276faf6e290 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:39.210927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0894b23d-7d27-4c21-97fb-8276faf6e290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:39.313965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 513/1025] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:23:39.597440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099247db-6355-459c-8ed4-9b9437ea0454 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:39.598987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-099247db-6355-459c-8ed4-9b9437ea0454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:39.655817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:39.702282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099247db-6355-459c-8ed4-9b9437ea0454 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:39.775633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:39.776232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:39.776510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:39.776778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:39.778383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:39.884195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:39.888293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:39.889751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26ffcf4-05ec-4a3c-a0de-af1bf9833ac7 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:39.907536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 513/1026] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:23:44.750500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33d6a1-62cf-409c-ac23-0dde188c0fad None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:44.750500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33d6a1-62cf-409c-ac23-0dde188c0fad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:44.750500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33d6a1-62cf-409c-ac23-0dde188c0fad None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:44.752245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 514/1027] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:44.763968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22759-f4bf-4909-8b3f-134efcebe3f9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:44.764494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22759-f4bf-4909-8b3f-134efcebe3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:44.764998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b22759-f4bf-4909-8b3f-134efcebe3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:44.891207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a9b22759-f4bf-4909-8b3f-134efcebe3f9 None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:44.899463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a9b22759-f4bf-4909-8b3f-134efcebe3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:45.046790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 514/1028] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:23:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:49.163579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce095-c319-4536-8c63-5c0d55c35226 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:49.164088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce095-c319-4536-8c63-5c0d55c35226 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:49.164088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce095-c319-4536-8c63-5c0d55c35226 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:49.165385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 515/1029] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:49.170929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d944e6-036e-4996-8781-ffdcd7f81814 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:49.171358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d944e6-036e-4996-8781-ffdcd7f81814 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:49.171506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d944e6-036e-4996-8781-ffdcd7f81814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:49.230526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:49.261347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-62d944e6-036e-4996-8781-ffdcd7f81814 None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:49.262843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-62d944e6-036e-4996-8781-ffdcd7f81814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:49.312261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=59 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:49.380927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 515/1030] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:23:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:50.683177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691976d-3fa2-43d8-9547-97a310b49941 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:50.683753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691976d-3fa2-43d8-9547-97a310b49941 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:50.683838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6691976d-3fa2-43d8-9547-97a310b49941 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:50.685143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 516/1031] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:50.690347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0f237-7e12-4baf-96b3-5b109df41194 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:50.690741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0f237-7e12-4baf-96b3-5b109df41194 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:50.691015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0f237-7e12-4baf-96b3-5b109df41194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:50.771622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-12d0f237-7e12-4baf-96b3-5b109df41194 None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:50.772817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-12d0f237-7e12-4baf-96b3-5b109df41194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:50.817037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:50.884658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 516/1032] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:23:50 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:54.519531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207bb70-9b86-4a59-8ffb-9be66f4a390a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:54.520054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207bb70-9b86-4a59-8ffb-9be66f4a390a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:54.520134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207bb70-9b86-4a59-8ffb-9be66f4a390a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:54.521554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 517/1033] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:54.526957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a42ff2-f2fc-4a5a-bd60-3396956e89fc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:54.527439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a42ff2-f2fc-4a5a-bd60-3396956e89fc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:54.527524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a42ff2-f2fc-4a5a-bd60-3396956e89fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:54.595696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-99a42ff2-f2fc-4a5a-bd60-3396956e89fc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:54.597448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-99a42ff2-f2fc-4a5a-bd60-3396956e89fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:54.717374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 517/1034] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:55.026859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2590-f9ee-449f-b6ad-6dd97b6f10d0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:55.027211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2590-f9ee-449f-b6ad-6dd97b6f10d0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:55.027409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0f2590-f9ee-449f-b6ad-6dd97b6f10d0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:55.028941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 518/1035] 10.222.0.17 () {54 vars in 961 bytes} [Wed Oct 12 12:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:55.037725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ee5ef-41af-4920-ad95-fc45720d053f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:55.038806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ee5ef-41af-4920-ad95-fc45720d053f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:55.039366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ee5ef-41af-4920-ad95-fc45720d053f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:55.060056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:55.130689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-706ee5ef-41af-4920-ad95-fc45720d053f None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:55.132929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-706ee5ef-41af-4920-ad95-fc45720d053f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:55.272187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 518/1036] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Oct 12 12:23:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:55.280071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2590-f9ee-449f-b6ad-6dd97b6f10d0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:55.281852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bd0f2590-f9ee-449f-b6ad-6dd97b6f10d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:55.436854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0f2590-f9ee-449f-b6ad-6dd97b6f10d0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:55.520582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:55.570524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:55.571182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:55.571502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:55.571778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:55.573491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:55.669482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:55.671446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccf9ae-ded5-41a8-968e-0294b92277f9 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:55.690634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 519/1037] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:23:58.078426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e5d5-a2a8-4a09-8609-568cebffa8a6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:58.078951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e5d5-a2a8-4a09-8609-568cebffa8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:58.079114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383e5d5-a2a8-4a09-8609-568cebffa8a6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:58.083465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 519/1038] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:23:58.085881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf362-0b89-49b3-950f-673060f28f32 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:58.086361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf362-0b89-49b3-950f-673060f28f32 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:58.086539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babf362-0b89-49b3-950f-673060f28f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:58.156533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4babf362-0b89-49b3-950f-673060f28f32 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:23:58.157790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4babf362-0b89-49b3-950f-673060f28f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:58.198774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:58.269136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 520/1039] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:23:58.567438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e5d5-a2a8-4a09-8609-568cebffa8a6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:23:58.569010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7383e5d5-a2a8-4a09-8609-568cebffa8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:58.654517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:58.754422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383e5d5-a2a8-4a09-8609-568cebffa8a6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:23:58.899287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:23:58.899916 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:23:58.900194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:23:58.900465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:23:58.902155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:23:58.926732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:23:59.001931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:23:59.003978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4bfc25-c2e4-452c-a30d-2e66fe36b62f service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:23:59.023317 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 520/1040] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:01.552817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b340-087f-4f3c-a066-c95b5e175ac9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:01.553373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b340-087f-4f3c-a066-c95b5e175ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:01.553373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082b340-087f-4f3c-a066-c95b5e175ac9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:01.554823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 521/1041] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:01.559979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1bb39-5dfd-4e2c-9fbd-d00e25f79557 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:01.560443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1bb39-5dfd-4e2c-9fbd-d00e25f79557 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:01.560667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1bb39-5dfd-4e2c-9fbd-d00e25f79557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:01.638012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bac1bb39-5dfd-4e2c-9fbd-d00e25f79557 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:01.639431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bac1bb39-5dfd-4e2c-9fbd-d00e25f79557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:01.748644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:01.757357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 521/1042] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:24:02.048352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dca43-f31a-41b2-8e3f-d0c2503d07d9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:02.048718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dca43-f31a-41b2-8e3f-d0c2503d07d9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:02.049704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dca43-f31a-41b2-8e3f-d0c2503d07d9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:02.051205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 522/1043] 10.222.0.17 () {54 vars in 961 bytes} [Wed Oct 12 12:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:02.056584 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21045e8a-16a5-4f5f-9253-dd8c2b88526c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:02.056872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21045e8a-16a5-4f5f-9253-dd8c2b88526c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:02.057143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21045e8a-16a5-4f5f-9253-dd8c2b88526c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:02.139410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-21045e8a-16a5-4f5f-9253-dd8c2b88526c None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:02.140607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-21045e8a-16a5-4f5f-9253-dd8c2b88526c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:02.271803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 522/1044] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Oct 12 12:24:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:02.279406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dca43-f31a-41b2-8e3f-d0c2503d07d9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:02.280924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa6dca43-f31a-41b2-8e3f-d0c2503d07d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:02.440054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dca43-f31a-41b2-8e3f-d0c2503d07d9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:02.461265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:02.568462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:02.569122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:02.569410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:02.569626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:02.571229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:02.628008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a53dc-d146-481f-b6eb-bf7ba52dc046 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:02.628537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a53dc-d146-481f-b6eb-bf7ba52dc046 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:02.628780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a53dc-d146-481f-b6eb-bf7ba52dc046 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:02.630401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 523/1045] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:24:02.635380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19551a9-6e16-470a-8f76-738e8264821b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:02.635663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19551a9-6e16-470a-8f76-738e8264821b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:02.635935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19551a9-6e16-470a-8f76-738e8264821b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:02.670655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:02.672477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a1760-fc4f-4874-a926-aa8c17bd049f service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:02.695644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 523/1046] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:02.709851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:02.718251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e19551a9-6e16-470a-8f76-738e8264821b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:02.719688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e19551a9-6e16-470a-8f76-738e8264821b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:02.844914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 524/1047] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:24:03.224304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924adc1e-c259-467e-883f-84cf26213621 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:03.224779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924adc1e-c259-467e-883f-84cf26213621 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:03.225215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924adc1e-c259-467e-883f-84cf26213621 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:03.227022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 524/1048] 10.222.0.17 () {54 vars in 963 bytes} [Wed Oct 12 12:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:03.232227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904292d-6f72-4968-9151-1e8c42771990 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:03.232517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904292d-6f72-4968-9151-1e8c42771990 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:03.232790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904292d-6f72-4968-9151-1e8c42771990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:03.318784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0904292d-6f72-4968-9151-1e8c42771990 None None] MFA Rules not processed for user `fde1aa258e9b400983bc8cffa1a8b99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:03.320001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0904292d-6f72-4968-9151-1e8c42771990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:03.396638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:03.405186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 525/1049] 10.222.0.17 () {58 vars in 1060 bytes} [Wed Oct 12 12:24:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:03.413272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924adc1e-c259-467e-883f-84cf26213621 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:03.414993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-924adc1e-c259-467e-883f-84cf26213621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:03.446152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:03.531839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924adc1e-c259-467e-883f-84cf26213621 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:03.618065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:03.618679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:03.618943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:03.619219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:03.620872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:03.726907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:03.787070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:03.790406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f51411-64d5-4dee-b316-8caeec3283b3 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:03.827889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 525/1050] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:04.915308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09d0c5-aa5e-482c-ba21-b2078d85aa10 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:04.915308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09d0c5-aa5e-482c-ba21-b2078d85aa10 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:04.915927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca09d0c5-aa5e-482c-ba21-b2078d85aa10 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:04.917015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 526/1051] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:04.922765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d904267-1fc2-44f1-9ded-6b23bc0c86d9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:04.923215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d904267-1fc2-44f1-9ded-6b23bc0c86d9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:04.923339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d904267-1fc2-44f1-9ded-6b23bc0c86d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:04.993149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9d904267-1fc2-44f1-9ded-6b23bc0c86d9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:04.994648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9d904267-1fc2-44f1-9ded-6b23bc0c86d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:05.091876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 526/1052] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:24:05.383737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09d0c5-aa5e-482c-ba21-b2078d85aa10 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:05.385411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca09d0c5-aa5e-482c-ba21-b2078d85aa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:05.545110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca09d0c5-aa5e-482c-ba21-b2078d85aa10 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:05.614790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:05.664617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:05.665222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:05.665527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:05.665796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:05.667406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:05.770365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:05.771965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba20a6-bb55-4815-a3be-6a6c22a6780c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:05.790818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 527/1053] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:08.026234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd2ff-8b90-4646-88ad-153137ec9ca4 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:08.026767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd2ff-8b90-4646-88ad-153137ec9ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:08.026767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dd2ff-8b90-4646-88ad-153137ec9ca4 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:08.028446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 527/1054] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:08.033443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcd2f0-15a7-45e5-ab86-57fbe62cfbce None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:08.034462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcd2f0-15a7-45e5-ab86-57fbe62cfbce None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:08.034603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dcd2f0-15a7-45e5-ab86-57fbe62cfbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:08.103466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-52dcd2f0-15a7-45e5-ab86-57fbe62cfbce None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:08.104647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-52dcd2f0-15a7-45e5-ab86-57fbe62cfbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:08.136039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:08.211260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 528/1055] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:08.500134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd2ff-8b90-4646-88ad-153137ec9ca4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:08.501904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-524dd2ff-8b90-4646-88ad-153137ec9ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:08.544313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:08.659965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dd2ff-8b90-4646-88ad-153137ec9ca4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:08.773697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:08.779447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:08.780465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:08.780812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:08.781152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:08.782832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:08.878096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:08.879905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffde8888-4921-4e47-97af-855f11426687 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:08.900574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 528/1056] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:11.125654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6d8e-5d22-4093-a6fc-3131e6807fc1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:11.126174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6d8e-5d22-4093-a6fc-3131e6807fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:11.126174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd6d8e-5d22-4093-a6fc-3131e6807fc1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:11.130862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 529/1057] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:11.133197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759cbae-5819-4709-99a7-4f41d05c09c2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:11.133654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759cbae-5819-4709-99a7-4f41d05c09c2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:11.133830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759cbae-5819-4709-99a7-4f41d05c09c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:11.204942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3759cbae-5819-4709-99a7-4f41d05c09c2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:11.206289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3759cbae-5819-4709-99a7-4f41d05c09c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:11.298352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:11.336750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 529/1058] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:11.633539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6d8e-5d22-4093-a6fc-3131e6807fc1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:11.635153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0cdd6d8e-5d22-4093-a6fc-3131e6807fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:11.791754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd6d8e-5d22-4093-a6fc-3131e6807fc1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:11.808330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:11.914855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:11.915719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:11.916009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:11.916290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:11.917950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:12.015733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:12.017808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6f1ef-734d-4579-bb03-02a5b434c10c service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:12.041263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 530/1059] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:14.338787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81217b3-141f-41b1-8336-7e6be344f409 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:14.339365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81217b3-141f-41b1-8336-7e6be344f409 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:14.339684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81217b3-141f-41b1-8336-7e6be344f409 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:14.341837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 530/1060] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:14.347325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498a93-97d1-407a-976e-7915702c730b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:14.347779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498a93-97d1-407a-976e-7915702c730b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:14.347886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a498a93-97d1-407a-976e-7915702c730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:14.369849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:14.418413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8a498a93-97d1-407a-976e-7915702c730b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:14.419703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8a498a93-97d1-407a-976e-7915702c730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:14.546371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 531/1061] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:14.834471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81217b3-141f-41b1-8336-7e6be344f409 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:14.836046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f81217b3-141f-41b1-8336-7e6be344f409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:15.010038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81217b3-141f-41b1-8336-7e6be344f409 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:15.066526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:15.135309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:15.136060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:15.136404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:15.136751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:15.138872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:15.240363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:15.242044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c2ae-160b-4083-bb55-c244d3d0cdaa service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:15.262015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 531/1062] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:17.498999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e537d-4f65-41e8-a2ad-c495034718b2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:17.499566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e537d-4f65-41e8-a2ad-c495034718b2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:17.499566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e537d-4f65-41e8-a2ad-c495034718b2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:17.500892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 532/1063] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:17.507290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d57082-6d83-4ce0-8daa-94d0683e7ded None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:17.507739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d57082-6d83-4ce0-8daa-94d0683e7ded None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:17.507974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d57082-6d83-4ce0-8daa-94d0683e7ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:17.575110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-71d57082-6d83-4ce0-8daa-94d0683e7ded None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:17.576217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-71d57082-6d83-4ce0-8daa-94d0683e7ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:17.584829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:17.673370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 532/1064] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:17.961907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e537d-4f65-41e8-a2ad-c495034718b2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:17.963441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f3e537d-4f65-41e8-a2ad-c495034718b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:18.055306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:18.111397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e537d-4f65-41e8-a2ad-c495034718b2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:18.227488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:18.228098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:18.228377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:18.228650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:18.230264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:18.296251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:18.325924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:18.327504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42fdee4-0ed1-4164-a0bd-524907d4d2b4 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:18.347079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 533/1065] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:19.123406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907c20-0d98-4e41-bc00-fbcf123c43cd None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:19.123955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907c20-0d98-4e41-bc00-fbcf123c43cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:19.123955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06907c20-0d98-4e41-bc00-fbcf123c43cd None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:19.125332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 533/1066] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:19.131677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92cbc8-5a4e-456a-b71a-c0a347bb1464 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:19.132102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92cbc8-5a4e-456a-b71a-c0a347bb1464 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:19.132224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db92cbc8-5a4e-456a-b71a-c0a347bb1464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:19.199446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-db92cbc8-5a4e-456a-b71a-c0a347bb1464 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:19.200551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-db92cbc8-5a4e-456a-b71a-c0a347bb1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:19.311760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 534/1067] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:19.614472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f01b-4a12-4b1f-a30a-9df3f1a6576c None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:19.614759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f01b-4a12-4b1f-a30a-9df3f1a6576c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:19.615032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa8f01b-4a12-4b1f-a30a-9df3f1a6576c None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:19.616355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 534/1068] 10.222.0.17 () {54 vars in 962 bytes} [Wed Oct 12 12:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:19.621678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb8e81-a413-43eb-8b35-f35135299216 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:19.622174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb8e81-a413-43eb-8b35-f35135299216 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:19.622457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eb8e81-a413-43eb-8b35-f35135299216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:19.668203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:19.699956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e4eb8e81-a413-43eb-8b35-f35135299216 None None] MFA Rules not processed for user `1fb121d7e44f4f868bc28dda9647bbee`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:19.701171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e4eb8e81-a413-43eb-8b35-f35135299216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:19.778159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 535/1069] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Oct 12 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:19.787864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f01b-4a12-4b1f-a30a-9df3f1a6576c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:19.789501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6aa8f01b-4a12-4b1f-a30a-9df3f1a6576c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:19.894289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa8f01b-4a12-4b1f-a30a-9df3f1a6576c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:19.932396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:19.968438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:19.969046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:19.969348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:19.969632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:19.971361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:20.080422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:20.082681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdb1a65-2815-4de9-8be4-70cffd3ad41f service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:20.101125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 535/1070] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:24:20.575730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769781c3-2192-48ce-9c16-22094a1ed383 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:20.576292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769781c3-2192-48ce-9c16-22094a1ed383 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:20.576292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769781c3-2192-48ce-9c16-22094a1ed383 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:20.577656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 536/1071] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:20.582873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b7fd4-33a2-4734-bb00-cddacdfd80c9 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:20.583298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b7fd4-33a2-4734-bb00-cddacdfd80c9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:20.583426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748b7fd4-33a2-4734-bb00-cddacdfd80c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:20.653378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-748b7fd4-33a2-4734-bb00-cddacdfd80c9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:20.654598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-748b7fd4-33a2-4734-bb00-cddacdfd80c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:20.674289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:20.763690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 536/1072] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:20.852852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:20.853203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:20.853608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:20.922508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:20.923702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:21.005074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:21.013213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 537/1073] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:21.059744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00c946-3f33-421c-94ac-00dc199026ea None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:21.060133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00c946-3f33-421c-94ac-00dc199026ea None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:21.060447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa00c946-3f33-421c-94ac-00dc199026ea None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:21.061856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 537/1074] 10.222.0.17 () {54 vars in 943 bytes} [Wed Oct 12 12:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:21.067128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:21.068334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:21.069751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:21.070036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:21.070309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:21.148576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] MFA Rules not processed for user `e7f71a2997894b309bd2333a972107d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:21.149740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:21.183995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02aa62-6a72-4279-ad6c-de02e9c34610 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:21.232799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:21.259707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 538/1075] 10.222.0.17 () {58 vars in 1040 bytes} [Wed Oct 12 12:24:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:21.270257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:21.280169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25aff7-0069-48b3-a943-a46dcb89e813 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:21.295271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:21.306675 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:21.307390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:21.307673 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:21.307939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:21.309549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:21.387407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:21.388012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:21.388293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:21.388566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:21.390259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:21.401565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:21.403105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f08ae-f8c6-4c19-9893-51e4d26f7316 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:21.421813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 538/1076] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:24:21.480285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:21.481920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29470861-c4cb-4718-bf64-de9a25a147af service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:21.490400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 539/1077] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:24:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:24:23.670320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116bbcc-45a9-48ff-8068-d14b1e84c14d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:23.670834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116bbcc-45a9-48ff-8068-d14b1e84c14d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:23.670834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8116bbcc-45a9-48ff-8068-d14b1e84c14d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:23.672144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 539/1078] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:23.678101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1fce-06fa-47b0-ac5f-bbd187e30c60 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:23.678875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1fce-06fa-47b0-ac5f-bbd187e30c60 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:23.679160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d1fce-06fa-47b0-ac5f-bbd187e30c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:23.692811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:23.757190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d87d1fce-06fa-47b0-ac5f-bbd187e30c60 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:23.758442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d87d1fce-06fa-47b0-ac5f-bbd187e30c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:23.864299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 540/1079] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:24.158641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116bbcc-45a9-48ff-8068-d14b1e84c14d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:24.160253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8116bbcc-45a9-48ff-8068-d14b1e84c14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:24.291557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:24.334169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8116bbcc-45a9-48ff-8068-d14b1e84c14d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:24.496175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:24.496910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:24.497424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:24.497867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:24.504984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:24.600072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:24.616094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:24.618750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c6e261-1f05-4747-91a2-aae8c92c2568 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:24.639731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 540/1080] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:26.970776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab02d4-d73c-463e-9887-a292692f3d93 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:26.971350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab02d4-d73c-463e-9887-a292692f3d93 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:26.971438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab02d4-d73c-463e-9887-a292692f3d93 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:26.977080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 541/1081] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:26.978677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5edf-16a5-4cc2-aaa5-ac804309afda None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:26.979138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5edf-16a5-4cc2-aaa5-ac804309afda None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:26.979239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5edf-16a5-4cc2-aaa5-ac804309afda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:27.055441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b37b5edf-16a5-4cc2-aaa5-ac804309afda None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:27.057157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b37b5edf-16a5-4cc2-aaa5-ac804309afda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:27.179041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 541/1082] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:24:27.469630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab02d4-d73c-463e-9887-a292692f3d93 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:27.471141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8eab02d4-d73c-463e-9887-a292692f3d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:27.573371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:27.641762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab02d4-d73c-463e-9887-a292692f3d93 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:27.770770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:27.771373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:27.771646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:27.771916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:27.773669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:27.825710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:27.870355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:27.872118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0c87b6-2c9a-4352-b70a-ef4d9522e851 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:27.892303 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 542/1083] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:30.098807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e119e4-b45e-496f-bf2f-82d80c9182d1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:30.099380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e119e4-b45e-496f-bf2f-82d80c9182d1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:30.099380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e119e4-b45e-496f-bf2f-82d80c9182d1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:30.101338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 542/1084] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:24:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:30.106104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5181-afb6-426b-840c-2b47967f0593 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:30.106522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5181-afb6-426b-840c-2b47967f0593 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:30.106698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb5181-afb6-426b-840c-2b47967f0593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:30.184955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1ccb5181-afb6-426b-840c-2b47967f0593 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:30.186230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1ccb5181-afb6-426b-840c-2b47967f0593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:30.297085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 543/1085] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:30.588316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e119e4-b45e-496f-bf2f-82d80c9182d1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:30.591678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c9e119e4-b45e-496f-bf2f-82d80c9182d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:30.654833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:30.751747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e119e4-b45e-496f-bf2f-82d80c9182d1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:30.883885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:30.884494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:30.884773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:30.885051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:30.886680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:30.907531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:30.989426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:30.991026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ebab4-7d9b-411b-bffb-3f395b7e15c9 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:31.012648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 543/1086] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:40.295412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:41.010205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=38 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:44.938370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef43274-2064-4c2a-9844-54fb0388f524 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:44.938827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef43274-2064-4c2a-9844-54fb0388f524 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:44.938930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef43274-2064-4c2a-9844-54fb0388f524 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:44.940268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 544/1087] 10.222.0.17 () {54 vars in 918 bytes} [Wed Oct 12 12:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:44.945369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf4b5-b570-4512-95d2-b4939ff75c95 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:44.945770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf4b5-b570-4512-95d2-b4939ff75c95 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:44.945917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bcf4b5-b570-4512-95d2-b4939ff75c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:45.025220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-31bcf4b5-b570-4512-95d2-b4939ff75c95 None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:45.026441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-31bcf4b5-b570-4512-95d2-b4939ff75c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:45.131932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 544/1088] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Oct 12 12:24:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:45.145937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e90a1b-7310-42dc-9d88-f918af938c59 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:45.146213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e90a1b-7310-42dc-9d88-f918af938c59 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:45.146480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e90a1b-7310-42dc-9d88-f918af938c59 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:45.147789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 545/1089] 10.222.0.17 () {54 vars in 918 bytes} [Wed Oct 12 12:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:45.152711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e332-a9ed-4801-adc7-ef6565c3d93b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:45.153000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e332-a9ed-4801-adc7-ef6565c3d93b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:45.153278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038e332-a9ed-4801-adc7-ef6565c3d93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:45.236695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:45.239875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8038e332-a9ed-4801-adc7-ef6565c3d93b None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:45.241618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8038e332-a9ed-4801-adc7-ef6565c3d93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:45.363163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 545/1090] 10.222.0.17 () {58 vars in 1015 bytes} [Wed Oct 12 12:24:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:47.957865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1228d-bd5e-4d21-80a8-a7102b28277f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:47.958435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1228d-bd5e-4d21-80a8-a7102b28277f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:47.958435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c1228d-bd5e-4d21-80a8-a7102b28277f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:47.959749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 546/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:24:47.964995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6b2fc-688a-4383-bd3e-1248daf6cc9c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:47.965497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6b2fc-688a-4383-bd3e-1248daf6cc9c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:47.965610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe6b2fc-688a-4383-bd3e-1248daf6cc9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:48.031844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ebe6b2fc-688a-4383-bd3e-1248daf6cc9c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:48.033070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ebe6b2fc-688a-4383-bd3e-1248daf6cc9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:48.086577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:48.128279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 546/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:48.415276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1228d-bd5e-4d21-80a8-a7102b28277f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:48.416886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-42c1228d-bd5e-4d21-80a8-a7102b28277f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:48.560729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c1228d-bd5e-4d21-80a8-a7102b28277f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:48.632589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:48.671376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173ef258-048c-456f-925b-79317683cf5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:48.671976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef258-048c-456f-925b-79317683cf5c service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:48.672250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef258-048c-456f-925b-79317683cf5c service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:48.672513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173ef258-048c-456f-925b-79317683cf5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:48.674127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-173ef258-048c-456f-925b-79317683cf5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:48.762921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef258-048c-456f-925b-79317683cf5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:48.764484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173ef258-048c-456f-925b-79317683cf5c service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:48.782790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 547/1093] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:49.919288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e53e5f-cbd2-4563-a257-5aafe2cf0b37 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:49.919852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e53e5f-cbd2-4563-a257-5aafe2cf0b37 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:49.919852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e53e5f-cbd2-4563-a257-5aafe2cf0b37 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:49.921170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 547/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:49.927531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ed4e-cab9-48a3-af1a-316a61742f2f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:49.927988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ed4e-cab9-48a3-af1a-316a61742f2f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:49.928086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081ed4e-cab9-48a3-af1a-316a61742f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:50.003887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2081ed4e-cab9-48a3-af1a-316a61742f2f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:50.005084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2081ed4e-cab9-48a3-af1a-316a61742f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:50.031870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:50.102261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 548/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:50.252231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e53e5f-cbd2-4563-a257-5aafe2cf0b37 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:50.254713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-11e53e5f-cbd2-4563-a257-5aafe2cf0b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:50.402871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e53e5f-cbd2-4563-a257-5aafe2cf0b37 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:50.442266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:50.514979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:50.515498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:50.515773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:50.516036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:50.517626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:50.610165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:50.611754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7210834-4542-4160-a01b-de7e01103487 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:50.630577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 548/1096] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:51.757152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43299d7b-9065-4a0c-aa0d-7cc1125c44ad None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:51.757764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43299d7b-9065-4a0c-aa0d-7cc1125c44ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:51.757764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43299d7b-9065-4a0c-aa0d-7cc1125c44ad None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:51.759125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 549/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:24:51.765384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348d50-252f-4485-af55-0ea38793af4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:51.765844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348d50-252f-4485-af55-0ea38793af4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:51.765953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca348d50-252f-4485-af55-0ea38793af4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:51.819276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:51.840889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ca348d50-252f-4485-af55-0ea38793af4f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:51.842108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca348d50-252f-4485-af55-0ea38793af4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:51.937322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 549/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:52.086188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43299d7b-9065-4a0c-aa0d-7cc1125c44ad None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:52.087650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-43299d7b-9065-4a0c-aa0d-7cc1125c44ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:52.234467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43299d7b-9065-4a0c-aa0d-7cc1125c44ad None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:52.246858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:52.349742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:52.350305 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:52.350584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:52.350854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:52.352449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:52.440129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:52.442341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f4adb6-4035-4383-a789-32d0b32dd66b service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:52.460575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 550/1099] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:53.560776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26e595-2c33-4ae6-ab0a-d0cd161e6bca None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:53.561401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26e595-2c33-4ae6-ab0a-d0cd161e6bca None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:53.561401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26e595-2c33-4ae6-ab0a-d0cd161e6bca None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:53.562589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 550/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:53.569258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85913076-50b7-45af-9d37-b9dc31a810f3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:53.569722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85913076-50b7-45af-9d37-b9dc31a810f3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:53.569835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85913076-50b7-45af-9d37-b9dc31a810f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:53.589466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:53.647665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-85913076-50b7-45af-9d37-b9dc31a810f3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:53.648862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-85913076-50b7-45af-9d37-b9dc31a810f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:53.742172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 551/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:53.891727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26e595-2c33-4ae6-ab0a-d0cd161e6bca None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:53.893163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6a26e595-2c33-4ae6-ab0a-d0cd161e6bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:54.002338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:54.038195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26e595-2c33-4ae6-ab0a-d0cd161e6bca None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:54.149206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:54.149762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:54.150037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:54.150307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:54.151889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:54.229319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:54.243308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:54.244905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9edd1a-8b4f-4210-829e-563023db2634 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:54.263306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 551/1102] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:55.373021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7edb1f-35ed-4617-aeef-7ed358b05158 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:55.373615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7edb1f-35ed-4617-aeef-7ed358b05158 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:55.373615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7edb1f-35ed-4617-aeef-7ed358b05158 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:55.374947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 552/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:55.381613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d505e9-dd00-4cde-b3d5-6fe16366285b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:55.382059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d505e9-dd00-4cde-b3d5-6fe16366285b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:55.382165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d505e9-dd00-4cde-b3d5-6fe16366285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:55.458339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b0d505e9-dd00-4cde-b3d5-6fe16366285b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:55.459535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b0d505e9-dd00-4cde-b3d5-6fe16366285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:55.556821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 552/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:55.707094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7edb1f-35ed-4617-aeef-7ed358b05158 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:55.708602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df7edb1f-35ed-4617-aeef-7ed358b05158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:55.792358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:55.854625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7edb1f-35ed-4617-aeef-7ed358b05158 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:55.966057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:55.966653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:55.966932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:55.967204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:55.968798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:56.018462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:56.061318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:56.062921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e0c36-a328-4f98-be93-e673b1893725 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:56.081256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 553/1105] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:24:57.136970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0d69c-fa68-47a3-b4ee-2cc155a8e920 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:57.137487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0d69c-fa68-47a3-b4ee-2cc155a8e920 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:57.137611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0d69c-fa68-47a3-b4ee-2cc155a8e920 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:57.138868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 553/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:57.146096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c160-9f95-4a27-91ce-ab6c0935a7eb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:57.146542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c160-9f95-4a27-91ce-ab6c0935a7eb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:57.146667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd64c160-9f95-4a27-91ce-ab6c0935a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:57.221655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bd64c160-9f95-4a27-91ce-ab6c0935a7eb None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:57.223160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bd64c160-9f95-4a27-91ce-ab6c0935a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:57.324960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 554/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:57.473640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0d69c-fa68-47a3-b4ee-2cc155a8e920 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:57.475086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-70e0d69c-fa68-47a3-b4ee-2cc155a8e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:57.523735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:57.620047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0d69c-fa68-47a3-b4ee-2cc155a8e920 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:57.731790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:57.732305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:57.732582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:57.732849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:57.734446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:57.742148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:57.823443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:57.825034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2275be-b666-453c-b3ee-3466c3e5f0f6 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:57.842939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 554/1108] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:24:58.898156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf2df5-e0b0-4321-8978-8ee3df057e06 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:58.898684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf2df5-e0b0-4321-8978-8ee3df057e06 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:58.898684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf2df5-e0b0-4321-8978-8ee3df057e06 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:58.899967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 555/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:24:58.906122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021326f9-a0fe-4095-a9bb-66c5d50b76e7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:58.906563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021326f9-a0fe-4095-a9bb-66c5d50b76e7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:58.906665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021326f9-a0fe-4095-a9bb-66c5d50b76e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:58.982518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-021326f9-a0fe-4095-a9bb-66c5d50b76e7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:24:58.983719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-021326f9-a0fe-4095-a9bb-66c5d50b76e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:59.051894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:59.080677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 555/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:24:59.230297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf2df5-e0b0-4321-8978-8ee3df057e06 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:24:59.231804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-15cf2df5-e0b0-4321-8978-8ee3df057e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:59.250273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:59.384461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf2df5-e0b0-4321-8978-8ee3df057e06 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:24:59.468431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:24:59.494233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:24:59.494766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:24:59.495041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:24:59.495337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:24:59.496918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:24:59.587528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:24:59.589100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f1d67e-d410-423f-b973-56ed664aaaec service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:24:59.607463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 556/1111] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:00.666588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2103510-29ec-4bc3-aa64-6442e8ce230b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:00.667184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2103510-29ec-4bc3-aa64-6442e8ce230b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:00.667184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2103510-29ec-4bc3-aa64-6442e8ce230b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:00.668450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 556/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:00.674866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc4bd-fffb-4864-954f-a0629d7987fd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:00.675900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc4bd-fffb-4864-954f-a0629d7987fd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:00.676318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdbc4bd-fffb-4864-954f-a0629d7987fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:00.752065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4bdbc4bd-fffb-4864-954f-a0629d7987fd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:00.753556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4bdbc4bd-fffb-4864-954f-a0629d7987fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:00.793908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:00.850002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 557/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:01.000883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2103510-29ec-4bc3-aa64-6442e8ce230b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:01.002451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2103510-29ec-4bc3-aa64-6442e8ce230b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:01.148855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2103510-29ec-4bc3-aa64-6442e8ce230b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:01.204883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:01.266833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:01.267740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:01.268029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:01.268303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:01.270641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:01.362074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:01.363780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beba521e-f44f-456a-ae14-10eb5398b76f service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:01.382409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 557/1114] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:02.430239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9ff43-25f0-4f27-8091-617fbe76c6e7 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:02.430802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9ff43-25f0-4f27-8091-617fbe76c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:02.430802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9ff43-25f0-4f27-8091-617fbe76c6e7 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:02.432124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 558/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:02.438162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e8e0-68df-4a6d-84e2-cebf89a6f1c9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:02.438575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e8e0-68df-4a6d-84e2-cebf89a6f1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:02.438741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d7e8e0-68df-4a6d-84e2-cebf89a6f1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:02.514103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b1d7e8e0-68df-4a6d-84e2-cebf89a6f1c9 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:02.515214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b1d7e8e0-68df-4a6d-84e2-cebf89a6f1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:02.517219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:02.612149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 558/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:25:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:02.762947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9ff43-25f0-4f27-8091-617fbe76c6e7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:02.764539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2be9ff43-25f0-4f27-8091-617fbe76c6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:02.907802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be9ff43-25f0-4f27-8091-617fbe76c6e7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:02.933795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:03.021449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:03.021974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:03.022247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:03.022516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:03.024120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:03.110657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:03.112237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ded6d-a73a-42a9-966d-411571229db1 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:03.130802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 559/1117] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:05.191571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5ae1-0855-4ae4-822c-529a3aebb00d None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:05.192109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5ae1-0855-4ae4-822c-529a3aebb00d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:05.192109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578a5ae1-0855-4ae4-822c-529a3aebb00d None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:05.193513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 559/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:05.198886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81446c4-b2a9-4527-a03b-27634fcb8003 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:05.199291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81446c4-b2a9-4527-a03b-27634fcb8003 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:05.199427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81446c4-b2a9-4527-a03b-27634fcb8003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:05.239559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:05.265623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d81446c4-b2a9-4527-a03b-27634fcb8003 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:05.266880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d81446c4-b2a9-4527-a03b-27634fcb8003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:05.363903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 560/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:05.650932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5ae1-0855-4ae4-822c-529a3aebb00d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:05.652460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-578a5ae1-0855-4ae4-822c-529a3aebb00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:05.754911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578a5ae1-0855-4ae4-822c-529a3aebb00d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:05.787375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:05.830427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:05.831027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:05.831329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:05.831607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:05.833175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:05.940437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:05.941866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470577c-0302-430c-9cf5-0e061fdae009 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:05.959879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 560/1120] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:08.267026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5789394-659f-4ffc-a091-120bc3848ee3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:08.267485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5789394-659f-4ffc-a091-120bc3848ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:08.267606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5789394-659f-4ffc-a091-120bc3848ee3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:08.268985 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 561/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:08.274412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefec28-3e60-4f74-8030-fcfcc93f7549 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:08.274855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefec28-3e60-4f74-8030-fcfcc93f7549 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:08.274997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefec28-3e60-4f74-8030-fcfcc93f7549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:08.340017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9fefec28-3e60-4f74-8030-fcfcc93f7549 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:08.341206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9fefec28-3e60-4f74-8030-fcfcc93f7549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:08.354139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:08.437416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 561/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:25:08.722770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5789394-659f-4ffc-a091-120bc3848ee3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:08.724310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e5789394-659f-4ffc-a091-120bc3848ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:08.826943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5789394-659f-4ffc-a091-120bc3848ee3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:08.840238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:08.900848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:08.901493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:08.901773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:08.902040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:08.903618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:09.012555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:09.014047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2085-4a26-423a-8335-ae51f2cc272a service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:09.032501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 562/1123] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:11.494340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc4f7-e561-4c28-8395-b421b1289ecd None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:11.494871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc4f7-e561-4c28-8395-b421b1289ecd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:11.494871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393cc4f7-e561-4c28-8395-b421b1289ecd None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:11.496232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 562/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:11.501820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c2815-22c9-408f-b389-da955e89a000 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:11.502264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c2815-22c9-408f-b389-da955e89a000 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:11.502362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c2815-22c9-408f-b389-da955e89a000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:11.552124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:11.571182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c82c2815-22c9-408f-b389-da955e89a000 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:11.572395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c82c2815-22c9-408f-b389-da955e89a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:11.668029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 563/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:11.951150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cc4f7-e561-4c28-8395-b421b1289ecd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:11.952755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-393cc4f7-e561-4c28-8395-b421b1289ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:12.056867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393cc4f7-e561-4c28-8395-b421b1289ecd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:12.095745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:12.131287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:12.131885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:12.132163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:12.132434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:12.134076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:12.242071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:12.243538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55797c71-9b7d-4646-a8af-57fc5b8cfc24 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:12.261814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 563/1126] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:14.539300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0e99f-d83b-4add-b52e-d978a7d86d90 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:14.539855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0e99f-d83b-4add-b52e-d978a7d86d90 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:14.539855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0e99f-d83b-4add-b52e-d978a7d86d90 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:14.541116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 564/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:14.546358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26373f-072d-4e2c-a271-304b86d6b7bd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:14.546761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26373f-072d-4e2c-a271-304b86d6b7bd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:14.546908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26373f-072d-4e2c-a271-304b86d6b7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:14.612519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3a26373f-072d-4e2c-a271-304b86d6b7bd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:14.614499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3a26373f-072d-4e2c-a271-304b86d6b7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:14.635453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:14.710310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 564/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:14.996600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0e99f-d83b-4add-b52e-d978a7d86d90 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:14.998182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-49a0e99f-d83b-4add-b52e-d978a7d86d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:15.100833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0e99f-d83b-4add-b52e-d978a7d86d90 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:15.121665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:15.176046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:15.176649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:15.176926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:15.177197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:15.178822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:15.288246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:15.289706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827229ff-f4c2-4b4b-b25a-184447bf9b48 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:15.307172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 565/1129] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:17.674792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075654b4-12cb-406e-aa0e-4797b8a5fc24 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:17.675831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075654b4-12cb-406e-aa0e-4797b8a5fc24 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:17.675831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075654b4-12cb-406e-aa0e-4797b8a5fc24 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:17.677530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 565/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:17.682845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b33e3-829f-4280-90f6-584d59db5079 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:17.683294 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b33e3-829f-4280-90f6-584d59db5079 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:17.683400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b33e3-829f-4280-90f6-584d59db5079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:17.749061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-528b33e3-829f-4280-90f6-584d59db5079 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:17.750331 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-528b33e3-829f-4280-90f6-584d59db5079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:17.750924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:17.848757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 566/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:18.135436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075654b4-12cb-406e-aa0e-4797b8a5fc24 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:18.137009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-075654b4-12cb-406e-aa0e-4797b8a5fc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:18.238413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075654b4-12cb-406e-aa0e-4797b8a5fc24 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:18.294926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:18.319041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:18.319638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:18.319911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:18.320177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:18.321798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:18.430458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:18.431809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0945f4d-be27-4955-88db-e373fa9f2402 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:18.449178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 566/1132] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:20.696464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4348a7c-0129-432b-a17d-d61e7aeb7bc3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:20.696998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4348a7c-0129-432b-a17d-d61e7aeb7bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:20.696998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4348a7c-0129-432b-a17d-d61e7aeb7bc3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:20.698314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 567/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:20.703537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b3a75-bfe9-4045-a8e6-40350dff9e21 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:20.703949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b3a75-bfe9-4045-a8e6-40350dff9e21 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:20.704086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b3a75-bfe9-4045-a8e6-40350dff9e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:20.785611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8d0b3a75-bfe9-4045-a8e6-40350dff9e21 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:20.786859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d0b3a75-bfe9-4045-a8e6-40350dff9e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:20.817798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:20.887692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 567/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:21.174515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4348a7c-0129-432b-a17d-d61e7aeb7bc3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:21.176147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e4348a7c-0129-432b-a17d-d61e7aeb7bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:21.278618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4348a7c-0129-432b-a17d-d61e7aeb7bc3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:21.307217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:21.351880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:21.352477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:21.352750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:21.353017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:21.354646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:21.462020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:21.463437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2e71f-613b-4c91-962c-814dc47f174e service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:21.481365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 568/1135] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:23.715481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e943d-5acb-45a6-b317-87f5acc041eb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:23.716021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e943d-5acb-45a6-b317-87f5acc041eb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:23.716021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e943d-5acb-45a6-b317-87f5acc041eb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:23.717286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 568/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:23.723834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c4b94-dfbe-44af-8895-bbb983adee06 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:23.724255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c4b94-dfbe-44af-8895-bbb983adee06 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:23.724378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c4b94-dfbe-44af-8895-bbb983adee06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:23.791273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-488c4b94-dfbe-44af-8895-bbb983adee06 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:23.792494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-488c4b94-dfbe-44af-8895-bbb983adee06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:23.801059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:23.888139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 569/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:24.172696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e943d-5acb-45a6-b317-87f5acc041eb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:24.174287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-196e943d-5acb-45a6-b317-87f5acc041eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:24.278707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e943d-5acb-45a6-b317-87f5acc041eb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:24.334780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:24.353137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:24.353777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:24.354054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:24.354323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:24.355983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:24.465023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:24.466490 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f14eb6-0efa-405f-9917-ce863a2925da service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:24.488729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 569/1138] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:26.749370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61250c9-7d87-40bc-978d-5b17b52a82e0 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:26.749934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61250c9-7d87-40bc-978d-5b17b52a82e0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:26.749934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61250c9-7d87-40bc-978d-5b17b52a82e0 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:26.751285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 570/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:26.756976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375934-9724-4f62-a8f4-8159bdfb5f8d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:26.757462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375934-9724-4f62-a8f4-8159bdfb5f8d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:26.757567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375934-9724-4f62-a8f4-8159bdfb5f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:26.823045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3f375934-9724-4f62-a8f4-8159bdfb5f8d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:26.824272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3f375934-9724-4f62-a8f4-8159bdfb5f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:26.862207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:26.920949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 570/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:27.200628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61250c9-7d87-40bc-978d-5b17b52a82e0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:27.202254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e61250c9-7d87-40bc-978d-5b17b52a82e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:27.313832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61250c9-7d87-40bc-978d-5b17b52a82e0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:27.351713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:27.388477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:27.389075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:27.389373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:27.389642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:27.391217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:27.500759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:27.502221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3e3c99-ccb5-49bb-b05c-87b7a8ac2ed4 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:27.521546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 571/1141] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:29.669658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018ae4f-e5c6-4f65-ad01-3942c89e12f2 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:29.670191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018ae4f-e5c6-4f65-ad01-3942c89e12f2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:29.670191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018ae4f-e5c6-4f65-ad01-3942c89e12f2 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:29.671460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 571/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:29.677509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb0c0-f4b2-47db-b47a-b5b0d08b3bd4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:29.677953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb0c0-f4b2-47db-b47a-b5b0d08b3bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:29.678054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bb0c0-f4b2-47db-b47a-b5b0d08b3bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:29.742691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c23bb0c0-f4b2-47db-b47a-b5b0d08b3bd4 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:29.743887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c23bb0c0-f4b2-47db-b47a-b5b0d08b3bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:29.761745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:29.837909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 572/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:30.120771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018ae4f-e5c6-4f65-ad01-3942c89e12f2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:30.122384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5018ae4f-e5c6-4f65-ad01-3942c89e12f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:30.224443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018ae4f-e5c6-4f65-ad01-3942c89e12f2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:30.291786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:30.298371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:30.298972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:30.299247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:30.299563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:30.301249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:30.410124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:30.411482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fba1dba-65ee-4170-9ba1-81e54714d20b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:30.429181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 572/1144] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:32.675570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2e2d3-1594-4257-9d61-e8066a09c6fe None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:32.676136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2e2d3-1594-4257-9d61-e8066a09c6fe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:32.676136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2e2d3-1594-4257-9d61-e8066a09c6fe None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:32.677989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 573/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:25:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:32.683309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f834233-d673-4afa-8abf-29575c3e122b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:32.683718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f834233-d673-4afa-8abf-29575c3e122b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:32.683848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f834233-d673-4afa-8abf-29575c3e122b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:32.748618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6f834233-d673-4afa-8abf-29575c3e122b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:32.749835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f834233-d673-4afa-8abf-29575c3e122b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:32.797993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:32.845849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 573/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:33.130231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2e2d3-1594-4257-9d61-e8066a09c6fe None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:33.131771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-08d2e2d3-1594-4257-9d61-e8066a09c6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:33.232653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2e2d3-1594-4257-9d61-e8066a09c6fe None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:33.280128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:33.307076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:33.307671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:33.307946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:33.308212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:33.309814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:33.418188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:33.419632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9d1887-3ac7-4ec3-90a4-7f53b9ad62a2 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:33.437711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 574/1147] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:42.844311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=22 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:43.435433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=66 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:53.354007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a47b-de5d-436d-a30e-898f118fa60b None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:53.354606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a47b-de5d-436d-a30e-898f118fa60b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:53.354606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2a47b-de5d-436d-a30e-898f118fa60b None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:53.355919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 574/1148] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:53.361847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb36141-b8b0-4ae0-9c2a-ea12762c256f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:53.362364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb36141-b8b0-4ae0-9c2a-ea12762c256f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:53.362364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb36141-b8b0-4ae0-9c2a-ea12762c256f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:53.426995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cfb36141-b8b0-4ae0-9c2a-ea12762c256f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:53.428189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cfb36141-b8b0-4ae0-9c2a-ea12762c256f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:53.523589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 575/1149] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:53.817849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d31-257a-4220-a2aa-5c7a663709e4 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:53.818237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d31-257a-4220-a2aa-5c7a663709e4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:53.818895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e57d31-257a-4220-a2aa-5c7a663709e4 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:53.820312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 575/1150] 10.222.0.17 () {54 vars in 962 bytes} [Wed Oct 12 12:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:53.825227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fd78c-8973-4c3b-ae7d-d912bd0e26f4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:53.825540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fd78c-8973-4c3b-ae7d-d912bd0e26f4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:53.825839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fd78c-8973-4c3b-ae7d-d912bd0e26f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:53.901102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7e5fd78c-8973-4c3b-ae7d-d912bd0e26f4 None None] MFA Rules not processed for user `1fb121d7e44f4f868bc28dda9647bbee`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:53.902340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7e5fd78c-8973-4c3b-ae7d-d912bd0e26f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:53.922188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:53.977612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 576/1151] 10.222.0.17 () {58 vars in 1059 bytes} [Wed Oct 12 12:25:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:25:53.985383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d31-257a-4220-a2aa-5c7a663709e4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:53.986896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-27e57d31-257a-4220-a2aa-5c7a663709e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:54.088484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e57d31-257a-4220-a2aa-5c7a663709e4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:54.161752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:54.162354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:54.162631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:54.162917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:54.164479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:54.218589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:54.275621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:54.277092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec85844-0935-4dca-bc6f-3ff71ea7f63d service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:54.295283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 576/1152] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:25:56.528360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bbb5c-9cfd-4c6d-8e43-41ea5b3df1ce None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:56.528888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bbb5c-9cfd-4c6d-8e43-41ea5b3df1ce None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:56.528888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204bbb5c-9cfd-4c6d-8e43-41ea5b3df1ce None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:56.530211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 577/1153] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:56.535401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a5033-ad27-433f-9ff9-620b75ace49a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:56.535814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a5033-ad27-433f-9ff9-620b75ace49a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:56.535942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a5033-ad27-433f-9ff9-620b75ace49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:56.601018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-472a5033-ad27-433f-9ff9-620b75ace49a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:56.602253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-472a5033-ad27-433f-9ff9-620b75ace49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:56.697072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 577/1154] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:56.971331 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bbb5c-9cfd-4c6d-8e43-41ea5b3df1ce None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:56.972856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-204bbb5c-9cfd-4c6d-8e43-41ea5b3df1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:57.100174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204bbb5c-9cfd-4c6d-8e43-41ea5b3df1ce None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:25:57.151729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:25:57.196790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:25:57.197415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:57.197691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:57.197957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:57.198583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:25:57.199710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:25:57.199710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:25:57.199710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:25:57.200920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb8b3ca-800b-431f-b1a4-225d3f64a88d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:25:57.221539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 578/1155] 10.222.0.17 () {56 vars in 1108 bytes} [Wed Oct 12 12:25:56 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:25:59.403518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2031a1-25b0-4aac-9613-7078dc19c586 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:59.404080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2031a1-25b0-4aac-9613-7078dc19c586 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:59.404080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2031a1-25b0-4aac-9613-7078dc19c586 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:59.405423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 578/1156] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:25:59.410757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ca5e6-47a1-41a2-ade1-d0580693b6fb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:25:59.411152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ca5e6-47a1-41a2-ade1-d0580693b6fb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:25:59.411307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ca5e6-47a1-41a2-ade1-d0580693b6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:25:59.475392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-431ca5e6-47a1-41a2-ade1-d0580693b6fb None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:25:59.476573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-431ca5e6-47a1-41a2-ade1-d0580693b6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:59.572427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 579/1157] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:25:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:25:59.870518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2031a1-25b0-4aac-9613-7078dc19c586 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:25:59.872174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b2031a1-25b0-4aac-9613-7078dc19c586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:25:59.882227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:00.018407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2031a1-25b0-4aac-9613-7078dc19c586 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:26:00.097324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:00.129962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:26:00.130494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:00.130769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:00.131038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:00.132613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:00.222915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:26:00.224496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1837ad77-8daf-4c6f-ac26-67c627d7c863 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:26:00.242252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 579/1158] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:25:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:26:02.579134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d67a98-1e18-43cb-b88b-04af2742b6f8 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:02.579665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d67a98-1e18-43cb-b88b-04af2742b6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:02.579665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d67a98-1e18-43cb-b88b-04af2742b6f8 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:02.580910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 580/1159] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:26:02.586328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fd211-54f5-462d-af6f-c524d08b79c4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:02.586799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fd211-54f5-462d-af6f-c524d08b79c4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:02.586905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fd211-54f5-462d-af6f-c524d08b79c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:02.652324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-013fd211-54f5-462d-af6f-c524d08b79c4 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:26:02.653556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-013fd211-54f5-462d-af6f-c524d08b79c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:02.698900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:02.750323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 580/1160] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:26:03.037392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d67a98-1e18-43cb-b88b-04af2742b6f8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:26:03.038916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e3d67a98-1e18-43cb-b88b-04af2742b6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:03.063152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:03.143386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d67a98-1e18-43cb-b88b-04af2742b6f8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:26:03.216152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:26:03.216680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:03.216957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:03.217226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:03.219012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:03.292795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:03.327889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:26:03.329736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a9d98-a9eb-445b-8d01-03867e3bcddc service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:26:03.347485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 581/1161] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:26:05.524892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93153798-3791-4e49-b669-81effc11bd4f None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:05.525453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93153798-3791-4e49-b669-81effc11bd4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:05.525453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93153798-3791-4e49-b669-81effc11bd4f None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:05.526709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 581/1162] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:26:05.532121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678e3f-b758-49af-bd0c-0e54e53fd34f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:05.532650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678e3f-b758-49af-bd0c-0e54e53fd34f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:05.532650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678e3f-b758-49af-bd0c-0e54e53fd34f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:05.598334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-12678e3f-b758-49af-bd0c-0e54e53fd34f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:26:05.599541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12678e3f-b758-49af-bd0c-0e54e53fd34f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:05.697126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 582/1163] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:26:05.984009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93153798-3791-4e49-b669-81effc11bd4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:26:05.986589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93153798-3791-4e49-b669-81effc11bd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:06.088448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93153798-3791-4e49-b669-81effc11bd4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:26:06.159292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:06.160752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:26:06.161443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:06.161902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:06.162345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:06.164179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:06.280531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:26:06.282691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853be6c-c6e6-436c-b3dc-0aa238157556 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:26:06.301239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 582/1164] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:26:08.617934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b325fa4-7b2a-4ff7-8877-a110f6662c33 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:08.618436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b325fa4-7b2a-4ff7-8877-a110f6662c33 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:08.618436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b325fa4-7b2a-4ff7-8877-a110f6662c33 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:08.619803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 583/1165] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:26:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:26:08.625950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ebe46-531d-4a1d-8c90-1af1560fca78 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:08.626392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ebe46-531d-4a1d-8c90-1af1560fca78 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:08.626597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ebe46-531d-4a1d-8c90-1af1560fca78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:08.693206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c23ebe46-531d-4a1d-8c90-1af1560fca78 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:26:08.694450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c23ebe46-531d-4a1d-8c90-1af1560fca78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:08.746376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:08.790927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 583/1166] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:26:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:26:09.084850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b325fa4-7b2a-4ff7-8877-a110f6662c33 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:26:09.086422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1b325fa4-7b2a-4ff7-8877-a110f6662c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:09.122611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:09.188769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b325fa4-7b2a-4ff7-8877-a110f6662c33 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:26:09.263372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:26:09.264009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:09.264291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:09.264568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:09.266791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:09.347513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:09.376854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:26:09.378318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b14088-02bb-42c1-b897-9ef168da1209 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:26:09.396971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 584/1167] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:26:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:26:11.739756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4eddb-63d9-4dec-a21a-2f7519fb04cc None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:11.740286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4eddb-63d9-4dec-a21a-2f7519fb04cc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:11.740405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4eddb-63d9-4dec-a21a-2f7519fb04cc None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:11.741764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 584/1168] 10.222.0.17 () {54 vars in 927 bytes} [Wed Oct 12 12:26:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:26:11.747137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa70065-a01b-4086-aae0-e14ef9afbfc2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:11.747542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa70065-a01b-4086-aae0-e14ef9afbfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:11.747682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa70065-a01b-4086-aae0-e14ef9afbfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:11.814537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-caa70065-a01b-4086-aae0-e14ef9afbfc2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:26:11.815732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-caa70065-a01b-4086-aae0-e14ef9afbfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:11.912336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 585/1169] 10.222.0.17 () {58 vars in 1024 bytes} [Wed Oct 12 12:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:26:12.199808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4eddb-63d9-4dec-a21a-2f7519fb04cc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:26:12.201388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-46f4eddb-63d9-4dec-a21a-2f7519fb04cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:12.306259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4eddb-63d9-4dec-a21a-2f7519fb04cc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:26:12.385387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:26:12.386028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:26:12.386306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:26:12.386576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:26:12.388176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:26:12.397463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:12.496582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:26:12.498036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68983f-6979-4e1e-abc9-198067b511e2 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:26:12.516669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 585/1170] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:26:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:26:21.910862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=82 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:26:22.513990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=50 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:24.986209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:24.986780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:24.986780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:25.063135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:27:25.064351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:25.161503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 586/1171] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:27:25.169575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:25.170072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:25.170173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:25.243988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:27:25.245148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:25.342385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 586/1172] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:27:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:27:25.347003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:25.348233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:25.359612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58886833-025c-4988-ab41-55dc077ab4a4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:25.420318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:25.453582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:25.454180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:25.454456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:25.454728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:25.455359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:27:25.455659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:25.456657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:27:25.456657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:27:25.456657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:27:25.457561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:25.469354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 587/1173] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:27:25.478942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:25.487949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecd53f6-e084-4a24-a822-013fc31dbff6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:25.550177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:25.584014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:25.584613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:25.584887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:25.585158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:25.585816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:25.586820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:27:25.586820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:27:25.586820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:27:25.588000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:25.643959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 587/1174] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:27:25 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:27:25.648160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:25.649700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:25.681914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc491d6-3cb0-40b1-890b-731670114222 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:25.775290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:25.775804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:25.776108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:25.776376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:25.776990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:25.778034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:27:25.778034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:27:25.778034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:27:25.779225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:25.822525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 588/1175] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:27:25 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:27:25.827188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:25.828835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:25.898778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eab6cf8-eb98-47ca-848e-1ca6472050f1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:25.986735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:26.000296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:26.000905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.001181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.001576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.002309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:26.003929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:26.016843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 588/1176] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:27:25 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:27:26.021030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:26.022599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:26.034495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d58d2-de31-49b0-aee7-45108e201d3a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:26.060729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:26.138197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a9811-9ce8-4de9-a7b2-df186c31d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:26.138659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a9811-9ce8-4de9-a7b2-df186c31d363 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.138937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a9811-9ce8-4de9-a7b2-df186c31d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.139205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a9811-9ce8-4de9-a7b2-df186c31d363 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.139851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a9811-9ce8-4de9-a7b2-df186c31d363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:26.141494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a9811-9ce8-4de9-a7b2-df186c31d363 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:26.154993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 589/1177] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:27:26.159239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:26.161073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:26.199112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97aa308-2838-446d-b5a2-2c5a5757ab80 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:26.316861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684b729-2598-4351-aa29-359d489479db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:26.317482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684b729-2598-4351-aa29-359d489479db admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.317762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684b729-2598-4351-aa29-359d489479db admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.318033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684b729-2598-4351-aa29-359d489479db admin admin] PATH_INFO: `/v3/projects/732b3bf106764228ba468a7c7d679e2d/users/6fef46d1e57c4e828b961d3ed283caf9/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.347886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684b729-2598-4351-aa29-359d489479db admin admin] RBAC: Authorizing `identity:create_grant(project_id=732b3bf106764228ba468a7c7d679e2d, user_id=6fef46d1e57c4e828b961d3ed283caf9, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:26.350719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4684b729-2598-4351-aa29-359d489479db admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:26.381277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 589/1178] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:27:26 2022] PUT /identity/v3/projects/732b3bf106764228ba468a7c7d679e2d/users/6fef46d1e57c4e828b961d3ed283caf9/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:27:26.393016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05996d5e-025a-48c2-b0a7-d6c79e529a20 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.393328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05996d5e-025a-48c2-b0a7-d6c79e529a20 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.393632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05996d5e-025a-48c2-b0a7-d6c79e529a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.460653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-05996d5e-025a-48c2-b0a7-d6c79e529a20 None None] MFA Rules not processed for user `6fef46d1e57c4e828b961d3ed283caf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:27:26.461903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-05996d5e-025a-48c2-b0a7-d6c79e529a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:26.561058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 590/1179] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:27:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:27:26.575954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71a147-0f01-4048-9821-939b26cfd530 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.576243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71a147-0f01-4048-9821-939b26cfd530 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.576521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71a147-0f01-4048-9821-939b26cfd530 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.577899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 590/1180] 10.222.0.17 () {54 vars in 963 bytes} [Wed Oct 12 12:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:27:26.584203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724432-d8ab-4c6e-9bfc-2d528dd46b7e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.584496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724432-d8ab-4c6e-9bfc-2d528dd46b7e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.584775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724432-d8ab-4c6e-9bfc-2d528dd46b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.623293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:26.676289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ef724432-d8ab-4c6e-9bfc-2d528dd46b7e None None] MFA Rules not processed for user `fde1aa258e9b400983bc8cffa1a8b99c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:27:26.678416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ef724432-d8ab-4c6e-9bfc-2d528dd46b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:26.754489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 591/1181] 10.222.0.17 () {58 vars in 1060 bytes} [Wed Oct 12 12:27:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:27:26.762886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71a147-0f01-4048-9821-939b26cfd530 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:26.764461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ba71a147-0f01-4048-9821-939b26cfd530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:26.814227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:26.868940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71a147-0f01-4048-9821-939b26cfd530 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:26.943629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:26.944226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:26.944500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:26.944766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:26.946385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:27.048132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:27.049579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:27.052933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:27.068494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 591/1182] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:27:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:27:27.115847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9b42a-dafb-4b9e-ad97-4b6a30473f07 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:27.116277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9b42a-dafb-4b9e-ad97-4b6a30473f07 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:27.116436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9b42a-dafb-4b9e-ad97-4b6a30473f07 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:27.117731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 592/1183] 10.222.0.17 () {52 vars in 830 bytes} [Wed Oct 12 12:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:27:27.122243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:27.123431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:27.224051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e270f-2d8a-4cc2-a73d-c5c1d28cb4b8 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:27.325038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:27.325715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:27.325987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:27.326268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] PATH_INFO: `/v3/users/6fef46d1e57c4e828b961d3ed283caf9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:27.339670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fef46d1e57c4e828b961d3ed283caf9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:27.340666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:27:27.340666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:27:27.340666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:27:27.341932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:27.351382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:27.408115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fef46d1e57c4e828b961d3ed283caf9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:27.408606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fef46d1e57c4e828b961d3ed283caf9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:27.409036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fef46d1e57c4e828b961d3ed283caf9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:27.422686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fef46d1e57c4e828b961d3ed283caf9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:27.432693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fef46d1e57c4e828b961d3ed283caf9'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:27.460697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 592/1184] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:27:27 2022] DELETE /identity/v3/users/6fef46d1e57c4e828b961d3ed283caf9 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:27:27.465888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:27.466172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:27.466446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:27.523519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:27:27.524731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:27.599568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:27.613468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 593/1185] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:27:27.624185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:27.625457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:27.726508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44996016-348c-4b10-adb0-b12cd4e725fc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:27.781802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:27.802881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:27.803489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:27.803766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:27.804035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:27.805683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:27.913703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:27.915144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803462f5-2e84-44ff-a38e-b176a3da8f89 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:27.934647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 593/1186] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:27:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:27:28.302962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:27:28.304184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:27:28.316591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1511ed5-5dd9-41f0-9f59-4bd770786093 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:27:28.416147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:27:28.416946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:27:28.417229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:27:28.417531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] PATH_INFO: `/v3/projects/732b3bf106764228ba468a7c7d679e2d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:27:28.428050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=732b3bf106764228ba468a7c7d679e2d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:27:28.429072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:27:28.429072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:27:28.429072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:27:28.430332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:27:28.487820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '732b3bf106764228ba468a7c7d679e2d', 'request_id': 'req-d4e6eb31-2ee9-4409-998b-02d35f225d6b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:28.501836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '732b3bf106764228ba468a7c7d679e2d', 'request_id': 'req-d4e6eb31-2ee9-4409-998b-02d35f225d6b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:28.512435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '732b3bf106764228ba468a7c7d679e2d', 'request_id': 'req-d4e6eb31-2ee9-4409-998b-02d35f225d6b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:28.535868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] The token cache is being invalidate because project 732b3bf106764228ba468a7c7d679e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:27:28.536169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:28.536666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4e6eb31-2ee9-4409-998b-02d35f225d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:27:28.537988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 594/1187] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:27:28 2022] DELETE /identity/v3/projects/732b3bf106764228ba468a7c7d679e2d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:27:37.932150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=62 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:27:38.533834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:28:57.510899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e23f95-51f7-48f4-ae18-dd65f92d4081 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:57.511498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e23f95-51f7-48f4-ae18-dd65f92d4081 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:57.511614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e23f95-51f7-48f4-ae18-dd65f92d4081 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:57.513026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 594/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:28:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:28:57.517845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784054-fa42-412b-8028-8d65016195b1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:57.518291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784054-fa42-412b-8028-8d65016195b1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:57.518398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21784054-fa42-412b-8028-8d65016195b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:57.597825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-21784054-fa42-412b-8028-8d65016195b1 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:28:57.599032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21784054-fa42-412b-8028-8d65016195b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:28:57.700540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 595/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:28:57.858336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e23f95-51f7-48f4-ae18-dd65f92d4081 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:28:57.860451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-16e23f95-51f7-48f4-ae18-dd65f92d4081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:28:57.980799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e23f95-51f7-48f4-ae18-dd65f92d4081 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:28:57.986195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322d4c1a-7286-4b29-9b4d-6b53d90071db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:28:57.986795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d4c1a-7286-4b29-9b4d-6b53d90071db service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:57.987072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d4c1a-7286-4b29-9b4d-6b53d90071db service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:57.987339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322d4c1a-7286-4b29-9b4d-6b53d90071db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:57.995625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d4c1a-7286-4b29-9b4d-6b53d90071db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:28:57.997237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322d4c1a-7286-4b29-9b4d-6b53d90071db service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:28:58.006592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 595/1190] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:28:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:28:59.096169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06492c0-b365-4f88-aca4-21ede9e786b9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:59.096665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06492c0-b365-4f88-aca4-21ede9e786b9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:59.096770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06492c0-b365-4f88-aca4-21ede9e786b9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:59.098111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 596/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:28:59.105075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580e00a-026d-4b28-be9c-bc2ac8d51a15 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:59.105576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580e00a-026d-4b28-be9c-bc2ac8d51a15 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:59.105672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5580e00a-026d-4b28-be9c-bc2ac8d51a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:59.125781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5580e00a-026d-4b28-be9c-bc2ac8d51a15 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:28:59.126995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5580e00a-026d-4b28-be9c-bc2ac8d51a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:28:59.134810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 596/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:28:59.285028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06492c0-b365-4f88-aca4-21ede9e786b9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:28:59.299733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06492c0-b365-4f88-aca4-21ede9e786b9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:28:59.305144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add5138e-9543-4195-9cd2-828dd2d7db9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:28:59.305777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5138e-9543-4195-9cd2-828dd2d7db9c service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:59.306054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5138e-9543-4195-9cd2-828dd2d7db9c service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:59.306321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add5138e-9543-4195-9cd2-828dd2d7db9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:59.315658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5138e-9543-4195-9cd2-828dd2d7db9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:28:59.317184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add5138e-9543-4195-9cd2-828dd2d7db9c service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:28:59.326592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 597/1193] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:28:59.332307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c60ff-8656-4afc-8a24-85f5264ca9f7 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:59.332307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c60ff-8656-4afc-8a24-85f5264ca9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:59.332522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6c60ff-8656-4afc-8a24-85f5264ca9f7 None None] PATH_INFO: `/v3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:59.333720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 597/1194] 10.222.0.43 () {54 vars in 925 bytes} [Wed Oct 12 12:28:59 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:28:59.340914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c889ad-74b9-4403-98a5-ba5718e2b757 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:28:59.341221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c889ad-74b9-4403-98a5-ba5718e2b757 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:28:59.341631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c889ad-74b9-4403-98a5-ba5718e2b757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:28:59.409535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c6c889ad-74b9-4403-98a5-ba5718e2b757 None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:28:59.410781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6c889ad-74b9-4403-98a5-ba5718e2b757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:28:59.429497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:28:59.486094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 598/1195] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Oct 12 12:28:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:29:08.175907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a998a66-350c-483a-ba9c-9a9695fb52e6 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:08.176359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a998a66-350c-483a-ba9c-9a9695fb52e6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:08.176467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a998a66-350c-483a-ba9c-9a9695fb52e6 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:08.177823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 598/1196] 10.222.0.17 () {54 vars in 961 bytes} [Wed Oct 12 12:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:08.182640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47044f-2a5f-45ee-a3c3-a0114f834b3b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:08.183085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47044f-2a5f-45ee-a3c3-a0114f834b3b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:08.183186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c47044f-2a5f-45ee-a3c3-a0114f834b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:08.203233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9c47044f-2a5f-45ee-a3c3-a0114f834b3b None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:08.204468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9c47044f-2a5f-45ee-a3c3-a0114f834b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:08.212895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 599/1197] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Oct 12 12:29:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:08.220541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a998a66-350c-483a-ba9c-9a9695fb52e6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:08.234215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a998a66-350c-483a-ba9c-9a9695fb52e6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:08.239153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afeb7b8-b07b-466e-a901-f23daf7b90c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:08.239737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeb7b8-b07b-466e-a901-f23daf7b90c5 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:08.240016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeb7b8-b07b-466e-a901-f23daf7b90c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:08.240286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afeb7b8-b07b-466e-a901-f23daf7b90c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:08.249198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeb7b8-b07b-466e-a901-f23daf7b90c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:08.250809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afeb7b8-b07b-466e-a901-f23daf7b90c5 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:08.260663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 599/1198] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:10.790249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247907ed-d8ff-4d63-bc98-cca1243cfd10 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:10.790791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247907ed-d8ff-4d63-bc98-cca1243cfd10 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:10.790791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247907ed-d8ff-4d63-bc98-cca1243cfd10 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:10.792129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 600/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:10.798318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f292-459b-4775-9833-4df89acc266c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:10.798788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f292-459b-4775-9833-4df89acc266c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:10.798898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd3f292-459b-4775-9833-4df89acc266c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:10.818495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bfd3f292-459b-4775-9833-4df89acc266c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:10.819685 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bfd3f292-459b-4775-9833-4df89acc266c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:10.827991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 600/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:10.983772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247907ed-d8ff-4d63-bc98-cca1243cfd10 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:10.999302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247907ed-d8ff-4d63-bc98-cca1243cfd10 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:11.004809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a51e5-4f51-4584-ad8a-9613c5894500 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:11.005426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a51e5-4f51-4584-ad8a-9613c5894500 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:11.005709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a51e5-4f51-4584-ad8a-9613c5894500 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:11.005982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a51e5-4f51-4584-ad8a-9613c5894500 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:11.014051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a51e5-4f51-4584-ad8a-9613c5894500 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:11.015621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a51e5-4f51-4584-ad8a-9613c5894500 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:11.024728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 601/1201] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:12.089763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359607d-27a1-47be-ac21-5e8a4b78b6de None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:12.090815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359607d-27a1-47be-ac21-5e8a4b78b6de None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:12.090935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359607d-27a1-47be-ac21-5e8a4b78b6de None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:12.092248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 601/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 12 12:29:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:12.098475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c855-bdcd-4db0-a2e0-c4599e42cfca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:12.098908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c855-bdcd-4db0-a2e0-c4599e42cfca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:12.099062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb67c855-bdcd-4db0-a2e0-c4599e42cfca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:12.118752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fb67c855-bdcd-4db0-a2e0-c4599e42cfca None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:12.120212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fb67c855-bdcd-4db0-a2e0-c4599e42cfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:12.128094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 602/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 12 12:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:12.282213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359607d-27a1-47be-ac21-5e8a4b78b6de None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:12.297040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359607d-27a1-47be-ac21-5e8a4b78b6de None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:12.302513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd1a1-51a2-4a75-b922-64597481d72b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:12.303100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd1a1-51a2-4a75-b922-64597481d72b service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:12.303379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd1a1-51a2-4a75-b922-64597481d72b service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:12.303650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd1a1-51a2-4a75-b922-64597481d72b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:12.311587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fd1a1-51a2-4a75-b922-64597481d72b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:12.313157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fd1a1-51a2-4a75-b922-64597481d72b service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:12.322430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 602/1204] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:16.397617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85f8e-52e0-44e8-84bc-fdf7ef8c2e5b None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:16.398141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85f8e-52e0-44e8-84bc-fdf7ef8c2e5b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:16.398261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a85f8e-52e0-44e8-84bc-fdf7ef8c2e5b None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:16.399532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 603/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:16.404804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6397e-dcfd-4ed1-8df6-2c7d3cb2c280 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:16.405199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6397e-dcfd-4ed1-8df6-2c7d3cb2c280 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:16.405386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6397e-dcfd-4ed1-8df6-2c7d3cb2c280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:16.424859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d4d6397e-dcfd-4ed1-8df6-2c7d3cb2c280 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:16.426082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d4d6397e-dcfd-4ed1-8df6-2c7d3cb2c280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:16.434406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 603/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:16.732764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85f8e-52e0-44e8-84bc-fdf7ef8c2e5b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:16.734558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f8a85f8e-52e0-44e8-84bc-fdf7ef8c2e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:16.811318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a85f8e-52e0-44e8-84bc-fdf7ef8c2e5b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:16.814856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b304bf4a-b964-4bc3-b8f2-888d2313ed86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:16.815449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304bf4a-b964-4bc3-b8f2-888d2313ed86 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:16.815722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304bf4a-b964-4bc3-b8f2-888d2313ed86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:16.815991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b304bf4a-b964-4bc3-b8f2-888d2313ed86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:16.824642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304bf4a-b964-4bc3-b8f2-888d2313ed86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:16.826088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b304bf4a-b964-4bc3-b8f2-888d2313ed86 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:16.835167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 604/1207] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:29:18.985376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f469bc1-fef8-46a2-b640-0a92625684b1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:18.985903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f469bc1-fef8-46a2-b640-0a92625684b1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:18.985999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f469bc1-fef8-46a2-b640-0a92625684b1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:18.987341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 604/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:18.992518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a648-ca2c-4462-96e6-3b47db76141a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:18.992941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a648-ca2c-4462-96e6-3b47db76141a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:18.993090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5801a648-ca2c-4462-96e6-3b47db76141a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:19.015458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5801a648-ca2c-4462-96e6-3b47db76141a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:19.016660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5801a648-ca2c-4462-96e6-3b47db76141a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:19.024521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 605/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:19.315512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f469bc1-fef8-46a2-b640-0a92625684b1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:19.320898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=75 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:19.327942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f469bc1-fef8-46a2-b640-0a92625684b1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:19.331477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac101f2-643e-4e63-9772-96e1056f056e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:19.332064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac101f2-643e-4e63-9772-96e1056f056e service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:19.332343 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac101f2-643e-4e63-9772-96e1056f056e service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:19.332612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac101f2-643e-4e63-9772-96e1056f056e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:19.342108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac101f2-643e-4e63-9772-96e1056f056e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:19.343539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac101f2-643e-4e63-9772-96e1056f056e service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:19.352742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 605/1210] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:22.939221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b981-5058-481b-83de-8fdee1a3bd82 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:22.939793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b981-5058-481b-83de-8fdee1a3bd82 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:22.939793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f79b981-5058-481b-83de-8fdee1a3bd82 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:22.941131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 606/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:22.946314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c34066-c3c5-4586-a27b-9797e08cfedd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:22.946727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c34066-c3c5-4586-a27b-9797e08cfedd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:22.946867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c34066-c3c5-4586-a27b-9797e08cfedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:22.966985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-26c34066-c3c5-4586-a27b-9797e08cfedd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:22.968189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-26c34066-c3c5-4586-a27b-9797e08cfedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:22.975973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 606/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:29:23.282896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b981-5058-481b-83de-8fdee1a3bd82 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:23.284639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4f79b981-5058-481b-83de-8fdee1a3bd82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:23.348248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:23.361861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f79b981-5058-481b-83de-8fdee1a3bd82 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:23.365460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855ba09-e9a3-4fef-9024-525032bbb68b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:23.366067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855ba09-e9a3-4fef-9024-525032bbb68b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:23.366345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855ba09-e9a3-4fef-9024-525032bbb68b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:23.366618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855ba09-e9a3-4fef-9024-525032bbb68b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:23.376213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855ba09-e9a3-4fef-9024-525032bbb68b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:23.377673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a855ba09-e9a3-4fef-9024-525032bbb68b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:23.387459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 607/1213] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:25.526025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809118-9b11-4eb8-b935-1e889229eccb None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:25.526573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809118-9b11-4eb8-b935-1e889229eccb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:25.526573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83809118-9b11-4eb8-b935-1e889229eccb None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:25.527948 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 607/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 12 12:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:29:25.533549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7239d0-144d-4b2f-bcd4-9e78585ceb67 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:25.534003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7239d0-144d-4b2f-bcd4-9e78585ceb67 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:25.534095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7239d0-144d-4b2f-bcd4-9e78585ceb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:25.554361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ee7239d0-144d-4b2f-bcd4-9e78585ceb67 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:29:25.555564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ee7239d0-144d-4b2f-bcd4-9e78585ceb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:25.564058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 608/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 12 12:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:29:25.864484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83809118-9b11-4eb8-b935-1e889229eccb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:29:25.866212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83809118-9b11-4eb8-b935-1e889229eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:29:25.879135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83809118-9b11-4eb8-b935-1e889229eccb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:29:25.882602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84922034-d7bb-4448-8b11-67212487e471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:29:25.883190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922034-d7bb-4448-8b11-67212487e471 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:29:25.883469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922034-d7bb-4448-8b11-67212487e471 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:29:25.883740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84922034-d7bb-4448-8b11-67212487e471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:29:25.892345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84922034-d7bb-4448-8b11-67212487e471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:29:25.893805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84922034-d7bb-4448-8b11-67212487e471 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:29:25.903427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 608/1216] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:29:35.549622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=11 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:29:35.898898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=14 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:50.560125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43f03-b894-420e-b9a6-02ccaeec138a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.560761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43f03-b894-420e-b9a6-02ccaeec138a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.560761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb43f03-b894-420e-b9a6-02ccaeec138a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.587480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-feb43f03-b894-420e-b9a6-02ccaeec138a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.589438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-feb43f03-b894-420e-b9a6-02ccaeec138a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.609353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 609/1217] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:50.610160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156f1e2-7874-4e1e-b416-2b97f59ee91c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.610698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156f1e2-7874-4e1e-b416-2b97f59ee91c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.610973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2156f1e2-7874-4e1e-b416-2b97f59ee91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.625612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.626003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.626229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.648820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2156f1e2-7874-4e1e-b416-2b97f59ee91c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.650065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2156f1e2-7874-4e1e-b416-2b97f59ee91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.661941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.663179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.670393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 609/1218] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:50.680967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.681334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.681555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.687834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 610/1219] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:50.693666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:50.709180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab1203-695d-4530-b3d8-bc93f7a7a44e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:50.715410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.716611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.729779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:50.730722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.731004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.731274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.732085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:50.732387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:50.732513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 610/1220] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:30:50.734190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5450a045-008f-42de-b80f-9dfb15ddc16d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:50.736861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:50.751265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba242be-2a6e-46f9-88af-466bd221e584 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:50.759090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:50.759726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.760004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.760274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.761554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:50.761859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:50.762950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:30:50.762950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:30:50.762950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:30:50.763849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:50.764114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 611/1221] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:30:50.770842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.771148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.771392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.786825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 611/1222] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:30:50.787506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:50.803765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f49a03-7724-4b1f-9daf-27b0af7d1f52 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:50.814224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe85dde-9615-4883-b93c-de60ac60132d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:50.814783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe85dde-9615-4883-b93c-de60ac60132d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.815066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe85dde-9615-4883-b93c-de60ac60132d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.815339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe85dde-9615-4883-b93c-de60ac60132d admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.815986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe85dde-9615-4883-b93c-de60ac60132d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:50.817989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe85dde-9615-4883-b93c-de60ac60132d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:50.826286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.827671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.852164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 612/1223] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:30:50.853043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:50.868955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1567d-9980-43d6-b055-16c473b8c90d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:50.877446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:50.878065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.878343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.878764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.879409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:50.880431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:30:50.880431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:30:50.880431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:30:50.881646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:50.887190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 612/1224] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:50.888498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:50.888787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:50.889061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:50.934425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:50.935658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:50.946855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 613/1225] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/projects => generated 391 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:50.947746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.052964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 613/1226] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.054216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.066810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18106bd6-04f0-4f45-8a8b-f5c74ef877e3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.068869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90631bc-f5bd-443d-8380-b86eebfe52cd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.074911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.077098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.078066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ceaecb-b9f2-4894-8781-2fda3787c339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.078500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.078798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.079165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceaecb-b9f2-4894-8781-2fda3787c339 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.079444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceaecb-b9f2-4894-8781-2fda3787c339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.079574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.079753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ceaecb-b9f2-4894-8781-2fda3787c339 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.080374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ceaecb-b9f2-4894-8781-2fda3787c339 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.080488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:30:51.080488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:30:51.080488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:30:51.081722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9b900-04b7-44e6-80aa-2aae08bf3103 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.082497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ceaecb-b9f2-4894-8781-2fda3787c339 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.109188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 614/1227] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/users => generated 296 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.111218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 614/1228] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:50 2022] POST /identity/v3/users => generated 287 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.111406 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3da77e-b4f6-40ec-8c47-608456fe68b2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.111849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3da77e-b4f6-40ec-8c47-608456fe68b2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.112390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3da77e-b4f6-40ec-8c47-608456fe68b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.114386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.114849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.115297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.141827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-df3da77e-b4f6-40ec-8c47-608456fe68b2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.142539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.143028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df3da77e-b4f6-40ec-8c47-608456fe68b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.143742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.151854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 615/1229] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.152507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 615/1230] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.153219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.159991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab3f72-a60e-4b5a-98c5-1ee617019581 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.160297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab3f72-a60e-4b5a-98c5-1ee617019581 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.160536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dab3f72-a60e-4b5a-98c5-1ee617019581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.163370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb584e7-1f89-4666-8c80-534f1b352d52 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.168626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.169232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.169671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.169954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.170601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:51.170902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.172511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013fc0b-3038-4e0c-b778-c63f9e930d9c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.180705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0dab3f72-a60e-4b5a-98c5-1ee617019581 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.181942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0dab3f72-a60e-4b5a-98c5-1ee617019581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.184318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 616/1231] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:51.185703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.185985 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.186259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.190731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 616/1232] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.192650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.192954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.193232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.206135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.207330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.214431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.215571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.217509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 617/1233] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.218309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.228028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 617/1234] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.228943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.231908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfc100-ec35-474c-8790-69bb621bce03 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.236660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca55bcdc-62c3-4ed2-be15-8c44b1d26a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.237355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55bcdc-62c3-4ed2-be15-8c44b1d26a46 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.237646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55bcdc-62c3-4ed2-be15-8c44b1d26a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.237922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca55bcdc-62c3-4ed2-be15-8c44b1d26a46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.238569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55bcdc-62c3-4ed2-be15-8c44b1d26a46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.239524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d6e19-8d4c-4474-a451-57091d9d9b70 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.240178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca55bcdc-62c3-4ed2-be15-8c44b1d26a46 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.244541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498c6ec-8f4d-485a-8101-2bd92aa0f32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.245380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498c6ec-8f4d-485a-8101-2bd92aa0f32c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.245759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498c6ec-8f4d-485a-8101-2bd92aa0f32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.246003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498c6ec-8f4d-485a-8101-2bd92aa0f32c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.246630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498c6ec-8f4d-485a-8101-2bd92aa0f32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.248274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498c6ec-8f4d-485a-8101-2bd92aa0f32c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.253971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 618/1235] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:51.253971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.253971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.253971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.261624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 618/1236] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:51.261624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b777616-1562-43a8-9737-a42d8e0bab27 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.261624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b777616-1562-43a8-9737-a42d8e0bab27 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.261624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b777616-1562-43a8-9737-a42d8e0bab27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.281725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.282101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2b777616-1562-43a8-9737-a42d8e0bab27 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.283068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.283258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2b777616-1562-43a8-9737-a42d8e0bab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.296530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 619/1237] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.296732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 619/1238] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.296829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.296907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f07c83-bb37-4e44-9d0b-3e58f3d3e740 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.296907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f07c83-bb37-4e44-9d0b-3e58f3d3e740 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.296907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f07c83-bb37-4e44-9d0b-3e58f3d3e740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.302639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5361-1c3d-4006-be36-e0219ab50fe1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.307395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3bf190-9ad2-4550-8e7d-5ba7bb9285de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.307987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3bf190-9ad2-4550-8e7d-5ba7bb9285de admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.308265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3bf190-9ad2-4550-8e7d-5ba7bb9285de admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.308541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3bf190-9ad2-4550-8e7d-5ba7bb9285de admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.309170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3bf190-9ad2-4550-8e7d-5ba7bb9285de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.311110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3bf190-9ad2-4550-8e7d-5ba7bb9285de admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.315499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-03f07c83-bb37-4e44-9d0b-3e58f3d3e740 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.316695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-03f07c83-bb37-4e44-9d0b-3e58f3d3e740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.324902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 620/1239] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.326084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.326370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.326648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.346253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 620/1240] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.346253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.346253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.346253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.346728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.347638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.365474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.366662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.392184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:51.394964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:51.414261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 621/1241] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.418268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.427395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6ecc8-4ea9-4d95-a711-b438d71dde43 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.432037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.433247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.433553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.433962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.434082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 621/1242] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.434709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.434816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.436444 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.443631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebd2a1-b3a6-47e1-afc3-2252541230f8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.447078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 622/1243] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:51.448133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.451539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.452143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.452426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.452699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.453368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.454975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.457886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005fdef-9a77-4d05-875f-5044d9d77d0d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.462533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.463109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.463391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.463719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.464352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:51.464682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.465219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 622/1244] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:51.465893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.466285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.475115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613019e9-9f94-4c85-b88a-f9de0be483ff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.480247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.480927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.481211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.488045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.488197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 623/1245] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:51.488310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:51.488400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.488482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.488482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.497353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 623/1246] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:51.498229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.498777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3151dd3-f0cb-4cd8-ba6d-80c450452da9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.504855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.505493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.505770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.506041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.506670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:51.506978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.507819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce0c52-2e29-43cd-8a15-ee2d19144a13 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.508556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.514593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.515176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.515451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.515721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.516339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.518247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.527741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 624/1247] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:51.527741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.531452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea112f1c-fe7a-4730-b6ef-ee6c6f1e3eba admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.536942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.537567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.537843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.538107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.538729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:51.539024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.540596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.542467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 624/1248] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.543596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.554279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 625/1249] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:51.555554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ae687d-60b3-4c1c-b1ba-155a7ac103ed admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.559433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.561252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.561956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.562242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.562518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.563192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:51.563501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.565108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.570438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfe67f1-e9ad-4895-b93b-528d04d6c39b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.576303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.576889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.577412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.577535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 625/1250] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:51.577619 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] PATH_INFO: `/v3/projects/63cab3964ec14361b50e591258284ab0/users/76e3059b01884215becf97694c3b7027/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.578063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.587287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea2201-6941-4d81-8844-211faee8a463 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.592482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.593133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.593432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.593702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] PATH_INFO: `/v3/projects/0798aa39fc88488882c81a36c176d5ff/users/fdae14d23d3e4eac8128fbefd4221be8/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.597618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] RBAC: Authorizing `identity:create_grant(project_id=63cab3964ec14361b50e591258284ab0, user_id=76e3059b01884215becf97694c3b7027, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.600293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.609528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0798aa39fc88488882c81a36c176d5ff, user_id=fdae14d23d3e4eac8128fbefd4221be8, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.612271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.615382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 626/1251] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:51 2022] PUT /identity/v3/projects/63cab3964ec14361b50e591258284ab0/users/76e3059b01884215becf97694c3b7027/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:51.616017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.629866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 626/1252] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:51 2022] PUT /identity/v3/projects/0798aa39fc88488882c81a36c176d5ff/users/fdae14d23d3e4eac8128fbefd4221be8/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:51.630661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.688403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dc1401-c4a6-4559-8c23-0908da08a44e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.692789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68533b45-3811-4114-9d0c-6b7b894e363a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.693028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.693668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.693946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.694214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.694841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.696748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.697522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.698745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.698745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.699296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.699833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.701767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.724207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 627/1253] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/projects => generated 365 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.725078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.729004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 627/1254] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.730872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.771455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:51.780608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:51.791089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750893b0-bab1-4e92-aea7-b984511440ba admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.796826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0974fefb-5b50-4141-b15a-077c59a43524 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.796987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.797505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.797811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.798084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.798715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.800629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.801516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.802114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.802394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.802665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.803304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.804927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.815463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 628/1255] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:51.816455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.826868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45620837-fa41-4ae6-be15-576b5b6aba00 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.829541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 628/1256] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.830422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:51.834588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c5ad7-c7d3-43e9-8fc4-61258d09f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.835181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5ad7-c7d3-43e9-8fc4-61258d09f276 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.835459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5ad7-c7d3-43e9-8fc4-61258d09f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.835727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c5ad7-c7d3-43e9-8fc4-61258d09f276 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.836351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c5ad7-c7d3-43e9-8fc4-61258d09f276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.838340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6c5ad7-c7d3-43e9-8fc4-61258d09f276 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.866023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 629/1257] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.867269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd3f5c-593f-4340-9da4-a26adf3cb80f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.867544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd3f5c-593f-4340-9da4-a26adf3cb80f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.867817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd3f5c-593f-4340-9da4-a26adf3cb80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.886717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7cdd3f5c-593f-4340-9da4-a26adf3cb80f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.887946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7cdd3f5c-593f-4340-9da4-a26adf3cb80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.894024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fc417d-95cf-4f2f-b246-832f80914d0b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:51.946898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 630/1258] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:51.950583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.950583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.950583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.952197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:51.952835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:51.953214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:51.953538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:51.954181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:51.956360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:51.976675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] MFA Rules not processed for user `fdae14d23d3e4eac8128fbefd4221be8`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:51.977898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:51.988233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 629/1259] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/projects => generated 373 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:51.989146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.046702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 631/1260] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:52.047353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.054903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4463825e-2f4e-453b-99b1-9ae3e59a2e56 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.057589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cdf6c0-95f0-4490-aa33-c4a9d4dd61cb None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.059800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.060399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.060734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.061012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.061760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.062113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.062709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.062987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.063259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.063714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.063886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.065839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.085266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:52.090213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 632/1261] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:52 2022] POST /identity/v3/users => generated 303 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:52.091114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.095562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 630/1262] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:51 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:52.097986 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.102811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1e047-a42e-4520-9b50-4bbea0665dc3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.108391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933ae99f-5d32-4108-b25f-5de9ebfc2d8c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.108743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.109398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.109674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.109943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.110575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.112217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.113007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.113665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.113946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.114223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.114848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.116785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.123355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 633/1263] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.124846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.135071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15218aa3-1622-4791-85a6-7a70eb0bf116 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.139524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 631/1264] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:52 2022] POST /identity/v3/users => generated 293 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:52.140341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.141751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.142021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:52.142411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.142662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.142917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.143656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.145886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.157474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:52.173861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 634/1265] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:52 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:52.173861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.181010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b24bc4-4ba0-478f-a4d4-d39a55c538a6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.185879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02766b8e-9cc5-4a95-8cdf-f3283c2fabc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.186472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02766b8e-9cc5-4a95-8cdf-f3283c2fabc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.186747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02766b8e-9cc5-4a95-8cdf-f3283c2fabc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.187015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02766b8e-9cc5-4a95-8cdf-f3283c2fabc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.187630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02766b8e-9cc5-4a95-8cdf-f3283c2fabc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.189573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02766b8e-9cc5-4a95-8cdf-f3283c2fabc0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.205555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c031000-cd60-4a77-adb1-60b30ffe60f1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.210298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.211786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.212672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.212672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.217949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 635/1266] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:52 2022] POST /identity/v3/users => generated 287 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:52.219303 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494033a-cf79-426e-8ec2-c5695795ec66 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.219736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494033a-cf79-426e-8ec2-c5695795ec66 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.220016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494033a-cf79-426e-8ec2-c5695795ec66 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.221385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 636/1267] 10.222.0.17 () {54 vars in 961 bytes} [Wed Oct 12 12:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:52.222132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494033a-cf79-426e-8ec2-c5695795ec66 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.223755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.225195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.234701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494033a-cf79-426e-8ec2-c5695795ec66 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.236036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 632/1268] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:52.236835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.239341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.239928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.240208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.240476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.241112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.242753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.246846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc250a-b366-4759-b3eb-f1fccbe3c94c service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.252711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.253353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.253681 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.253886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.254465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 637/1269] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.254638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.255377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.256239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.270816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2037a47-58e6-42ee-bb57-6c64662e2eee admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.273792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 633/1270] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.274724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.277032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.277751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.278030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.278301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] PATH_INFO: `/v3/projects/673075fbfa05499fbb33fbda768ff128/users/f9c1340a44044abdb254038f2f32a3d0/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.286296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a4c745-37a8-4709-b07b-92cd056931f9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.291170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.292574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.293246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.293558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.294199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.294827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=673075fbfa05499fbb33fbda768ff128, user_id=f9c1340a44044abdb254038f2f32a3d0, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.295847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.298393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.306467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 634/1271] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.307383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.317600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 638/1272] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:52 2022] PUT /identity/v3/projects/673075fbfa05499fbb33fbda768ff128/users/f9c1340a44044abdb254038f2f32a3d0/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:52.318356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebcab6e-6b08-4d35-9ed1-fed9567028a2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.318976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.412365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9ed9-7204-4ddd-acad-8360b19584fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.413009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9ed9-7204-4ddd-acad-8360b19584fd admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.413347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9ed9-7204-4ddd-acad-8360b19584fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.413552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9ed9-7204-4ddd-acad-8360b19584fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.414245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb9ed9-7204-4ddd-acad-8360b19584fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.415799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb9ed9-7204-4ddd-acad-8360b19584fd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.422171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948bc985-500b-415b-b5e7-3209d6d7b7d2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.427993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 635/1273] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.429839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.430193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.430949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.440631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.441568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.441947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.442341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.443061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.444996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.459488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 639/1274] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.460501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.475470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fa85f-7405-4615-84ff-cbdfbe93b431 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.479579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:52.480801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:52.481551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.482338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.482775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.483315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.484113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.486012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.497804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 640/1275] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.498860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.509864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7d810-8fe3-4ba1-aab1-6559795a781c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.516127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.516764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.517046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.517336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.518053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.519763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.533608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 641/1276] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.534290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.535995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:52.545449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389f6c9-b6fe-4b1c-ab79-ae47c88c4cba admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.551456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c6ee84-445a-43f5-8dc7-486d3b03eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.552050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ee84-445a-43f5-8dc7-486d3b03eec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.552328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ee84-445a-43f5-8dc7-486d3b03eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.552598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c6ee84-445a-43f5-8dc7-486d3b03eec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.553228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6ee84-445a-43f5-8dc7-486d3b03eec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.554858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c6ee84-445a-43f5-8dc7-486d3b03eec9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.571174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 642/1277] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.572686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.573097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.574030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.578524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 636/1278] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Oct 12 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:52.579764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.591428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251ebc4-b898-412b-a812-aa155828f7d0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.596645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.597376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.597863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.597988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.598734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.600441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.613002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] MFA Rules not processed for user `f9c1340a44044abdb254038f2f32a3d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:52.616387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 637/1279] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.616387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.618540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:52.626425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a6075-f87d-44c9-aa98-ccbac37cca6d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.631453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.632161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.632515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.632893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.633701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.635375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.646385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 638/1280] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:52.646385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.660663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336ec5b-e828-4344-9fc0-9f1a8d0a4d77 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.667922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.668163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.668337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.668714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] PATH_INFO: `/v3/projects/fe8cfe97aa2d474d8c78821b297c6c80/users/3391253ce97247c5a45ddcbdcc06e935/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.690885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8cfe97aa2d474d8c78821b297c6c80, user_id=3391253ce97247c5a45ddcbdcc06e935, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.693850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.696701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:52.709464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 643/1281] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:52.710363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.711199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 639/1282] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:52 2022] PUT /identity/v3/projects/fe8cfe97aa2d474d8c78821b297c6c80/users/3391253ce97247c5a45ddcbdcc06e935/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:52.712388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.737538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff04ff3-28c0-4e14-8654-6ce66910abd2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.787523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a4ffe-f9df-4a93-91e4-a25c1203aa47 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.793373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.794114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.794397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.794672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] PATH_INFO: `/v3/projects/557de755e1e8423f9ba4e1be0627ca53/users/383004527ce84e4f9fc40760a040936e/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.810509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=557de755e1e8423f9ba4e1be0627ca53, user_id=383004527ce84e4f9fc40760a040936e, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.813232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.814093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.815056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.815056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.815309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] PATH_INFO: `/v3/projects/3c899a3604654d39b0b25659c83a3c25/users/594363cac9524600a7be433905ee3d0c/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.830238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 640/1283] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:52 2022] PUT /identity/v3/projects/557de755e1e8423f9ba4e1be0627ca53/users/383004527ce84e4f9fc40760a040936e/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:52.831106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.835659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c899a3604654d39b0b25659c83a3c25, user_id=594363cac9524600a7be433905ee3d0c, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.838432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:52.853621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 644/1284] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:52 2022] PUT /identity/v3/projects/3c899a3604654d39b0b25659c83a3c25/users/594363cac9524600a7be433905ee3d0c/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:52.856169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:52.928310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e91c34-7ca1-4309-967a-d52f4da99e56 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.939352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:52.951347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b747554-e5e7-4193-9960-d0d3bf8f0526 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:52.959667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:52.960375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:52.960634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:52.961110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] PATH_INFO: `/v3/projects/c899460f7a66440b9321189597303fc9/users/25119286009743bbac4217d5faa40177/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:52.981492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c899460f7a66440b9321189597303fc9, user_id=25119286009743bbac4217d5faa40177, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:52.984321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.006578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 645/1285] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:52 2022] PUT /identity/v3/projects/c899460f7a66440b9321189597303fc9/users/25119286009743bbac4217d5faa40177/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:53.011025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.028626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e2186-510a-463c-a9fd-cf70c05a409f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.031059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2186-510a-463c-a9fd-cf70c05a409f service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.031059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2186-510a-463c-a9fd-cf70c05a409f service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.031059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e2186-510a-463c-a9fd-cf70c05a409f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.087493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1e68b-a73f-4a99-a94c-0bcb950b1964 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.094087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f0bb26-2300-4588-a13b-ad8dcfd98d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.094843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bb26-2300-4588-a13b-ad8dcfd98d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.095225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bb26-2300-4588-a13b-ad8dcfd98d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.095567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f0bb26-2300-4588-a13b-ad8dcfd98d3c admin admin] PATH_INFO: `/v3/projects/fa83f5942d9e44068e351f1256065ab7/users/abbc8a34ac8f44328596102c35e184dc/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.098478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:53.098660 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2186-510a-463c-a9fd-cf70c05a409f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.100758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e2186-510a-463c-a9fd-cf70c05a409f service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.118095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0bb26-2300-4588-a13b-ad8dcfd98d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa83f5942d9e44068e351f1256065ab7, user_id=abbc8a34ac8f44328596102c35e184dc, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.121819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f0bb26-2300-4588-a13b-ad8dcfd98d3c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.137736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 641/1286] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.138286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 646/1287] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:53 2022] PUT /identity/v3/projects/fa83f5942d9e44068e351f1256065ab7/users/abbc8a34ac8f44328596102c35e184dc/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:53.139668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f899-944f-4402-88b5-c7757e14a980 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.140026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f899-944f-4402-88b5-c7757e14a980 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.140383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420f899-944f-4402-88b5-c7757e14a980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.140788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aa7b2-b258-4f0e-a277-061cc2aabc1f None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.141344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aa7b2-b258-4f0e-a277-061cc2aabc1f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.141877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aa7b2-b258-4f0e-a277-061cc2aabc1f None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.143751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 647/1288] 10.222.0.17 () {54 vars in 961 bytes} [Wed Oct 12 12:30:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:53.145127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0008c4-f888-4daf-8736-7e2cdcd2516a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.145631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0008c4-f888-4daf-8736-7e2cdcd2516a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.146103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0008c4-f888-4daf-8736-7e2cdcd2516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.169221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5d0008c4-f888-4daf-8736-7e2cdcd2516a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.171083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5d0008c4-f888-4daf-8736-7e2cdcd2516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.176365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1420f899-944f-4402-88b5-c7757e14a980 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.177585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1420f899-944f-4402-88b5-c7757e14a980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.255427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 642/1289] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.267844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.268136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.268411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.283469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 648/1290] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.291922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188c85c-8355-4e9d-bf32-435be6abb4d6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.292662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188c85c-8355-4e9d-bf32-435be6abb4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.293124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188c85c-8355-4e9d-bf32-435be6abb4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.300863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] MFA Rules not processed for user `594363cac9524600a7be433905ee3d0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.303372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.330375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7188c85c-8355-4e9d-bf32-435be6abb4d6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.331989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7188c85c-8355-4e9d-bf32-435be6abb4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.342927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 649/1291] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.345137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d0cd-5a52-47ce-94ef-cca9caf5b575 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.357828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d0cd-5a52-47ce-94ef-cca9caf5b575 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.359636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201d0cd-5a52-47ce-94ef-cca9caf5b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.381500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:53.387739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4201d0cd-5a52-47ce-94ef-cca9caf5b575 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.389602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4201d0cd-5a52-47ce-94ef-cca9caf5b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.399391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 650/1292] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.400978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.402447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.409326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.432154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.436752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 643/1293] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.438405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.438700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.522603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265260d-68dc-4332-af73-30ba065d2a93 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.528529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.529236 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.529645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.530067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.538158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 651/1294] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.538345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.549100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70527e6-ff1c-47d2-8711-69ed71bb5c8e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.550675 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.552389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.553749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.554865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.555218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.555559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.564371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 644/1295] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.565347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.574838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd23c179-a373-410e-ace1-fe1b8673ac87 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.579206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.580673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.586320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aac38d-ecd3-4392-afa4-eafaa801af76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.587265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aac38d-ecd3-4392-afa4-eafaa801af76 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.587546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aac38d-ecd3-4392-afa4-eafaa801af76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.587923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aac38d-ecd3-4392-afa4-eafaa801af76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.594108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 652/1296] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.594984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.602068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aac38d-ecd3-4392-afa4-eafaa801af76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.603541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aac38d-ecd3-4392-afa4-eafaa801af76 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.605781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe32ffc-da9e-48c0-a4d0-595febe292d6 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.610339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.611086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.611522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.611824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.629210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 645/1297] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.630563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc3d14-c303-4dd0-aa02-3e23030315e1 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.630883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc3d14-c303-4dd0-aa02-3e23030315e1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.631167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc3d14-c303-4dd0-aa02-3e23030315e1 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.638151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 646/1298] 10.222.0.17 () {54 vars in 912 bytes} [Wed Oct 12 12:30:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:53.639283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.640948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.643187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc3d14-c303-4dd0-aa02-3e23030315e1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.664551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 653/1299] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.668693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.700238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4e0471-7862-4019-a62a-08c79ccb204b service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.711155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07af053-6f5d-47e6-9bf9-adee355138fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.712437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af053-6f5d-47e6-9bf9-adee355138fa service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.712865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af053-6f5d-47e6-9bf9-adee355138fa service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.713424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07af053-6f5d-47e6-9bf9-adee355138fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.767122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:53.777789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc3d14-c303-4dd0-aa02-3e23030315e1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.782649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.783342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.783564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.783802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.801692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.803150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.811685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07af053-6f5d-47e6-9bf9-adee355138fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:53.813527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07af053-6f5d-47e6-9bf9-adee355138fa service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:53.827977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 654/1300] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.829507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75220af0-ad33-469b-ba76-03bc54297b8c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.830100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75220af0-ad33-469b-ba76-03bc54297b8c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.830412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75220af0-ad33-469b-ba76-03bc54297b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.851920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 647/1301] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:53.876966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-75220af0-ad33-469b-ba76-03bc54297b8c None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:53.878357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75220af0-ad33-469b-ba76-03bc54297b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:53.898290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 655/1302] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:53.914391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:53.924976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a6d2dc-4a0e-4a6f-ba4d-9bf8224a2836 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:53.954053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8444768-674f-41f2-9eac-8d9da68c6b70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:53.954694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444768-674f-41f2-9eac-8d9da68c6b70 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:53.954974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444768-674f-41f2-9eac-8d9da68c6b70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:53.955248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8444768-674f-41f2-9eac-8d9da68c6b70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:53.996251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8444768-674f-41f2-9eac-8d9da68c6b70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:54.004467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8444768-674f-41f2-9eac-8d9da68c6b70 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:54.026919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 648/1303] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:54.488830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11adc9-5c59-400c-a522-a04e5071ff3d None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:54.489198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11adc9-5c59-400c-a522-a04e5071ff3d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:54.489507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11adc9-5c59-400c-a522-a04e5071ff3d None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:54.491638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 656/1304] 10.222.0.17 () {54 vars in 912 bytes} [Wed Oct 12 12:30:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:30:54.498129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:54.498429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:54.498701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:54.540367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:54.550863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:54.572378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 649/1305] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:54.762837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11adc9-5c59-400c-a522-a04e5071ff3d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:54.785752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11adc9-5c59-400c-a522-a04e5071ff3d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:54.791555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e2112a-0748-4f19-8bf7-53892d01492e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:54.792593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2112a-0748-4f19-8bf7-53892d01492e service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:54.793067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2112a-0748-4f19-8bf7-53892d01492e service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:54.793568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e2112a-0748-4f19-8bf7-53892d01492e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:54.816205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2112a-0748-4f19-8bf7-53892d01492e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:54.818014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e2112a-0748-4f19-8bf7-53892d01492e service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:54.843082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 657/1306] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:55.247091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:55.264727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78503480-4ef8-4c2c-b259-eaa052af4cdd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:55.284234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:55.284952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:55.285597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:55.286063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] PATH_INFO: `/v3/users/f9c1340a44044abdb254038f2f32a3d0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:55.289292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9c1340a44044abdb254038f2f32a3d0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:55.291699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:55.427722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9c1340a44044abdb254038f2f32a3d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:55.429909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9c1340a44044abdb254038f2f32a3d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:55.431621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9c1340a44044abdb254038f2f32a3d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:55.434610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:55.452698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9c1340a44044abdb254038f2f32a3d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:55.479783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9c1340a44044abdb254038f2f32a3d0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:55.513637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 650/1307] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:30:55 2022] DELETE /identity/v3/users/f9c1340a44044abdb254038f2f32a3d0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:55.518832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73d83a-f2aa-463a-b360-5705e9b2d6f0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:55.519192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73d83a-f2aa-463a-b360-5705e9b2d6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:55.519446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae73d83a-f2aa-463a-b360-5705e9b2d6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:55.566525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ae73d83a-f2aa-463a-b360-5705e9b2d6f0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:55.567782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae73d83a-f2aa-463a-b360-5705e9b2d6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:55.679386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 658/1308] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:55.742019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:55.744508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:55.809644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b00187a-1dfc-456a-b753-816a18e9a0b5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:55.813689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:55.814412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:55.814822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:55.815189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:55.826190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:55.827704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:55.838517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 651/1309] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:30:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:30:57.186566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.187088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.187304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.247360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] MFA Rules not processed for user `3391253ce97247c5a45ddcbdcc06e935`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:57.248756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:57.402365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 659/1310] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:57.407106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:57.409800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:57.436426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9cc68-6de0-4d87-8772-a198710b2f66 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:57.453937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:57.454921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.455208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.455506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.456118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:57.458080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:57.519894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 652/1311] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:57 2022] POST /identity/v3/projects => generated 403 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:57.523953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:57.551675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1064b00-e538-4431-8eab-da8db5014216 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:57.652671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:57.705955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:57.706598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.706881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.707164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.709432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:57.716784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:57.771520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 660/1312] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:57 2022] POST /identity/v3/users => generated 308 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:57.776734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:57.795355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a3610-e0e5-4d64-ad2d-b73275a8122f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:57.809866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:57.810783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.812731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.813164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.814080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:57.815878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:57.837456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 653/1313] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:57.839196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:57.859743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fedbeb-df59-488e-8997-9aed24b56b03 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:57.864599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:57.865279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:57.865614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:57.865883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] PATH_INFO: `/v3/projects/cd8a92c86f0845559187fcd2bbff07ea/users/75f64ef3503c42539533c0710055f1a7/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:57.900505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8a92c86f0845559187fcd2bbff07ea, user_id=75f64ef3503c42539533c0710055f1a7, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:57.904246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:57.931006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 661/1314] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:57 2022] PUT /identity/v3/projects/cd8a92c86f0845559187fcd2bbff07ea/users/75f64ef3503c42539533c0710055f1a7/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:57.935992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:57.964314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b451752-f38a-4a0e-a95f-f414194324d3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:57.990718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:57.992971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:58.162326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:58.162745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.163030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.163307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.163972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:58.165904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:58.186024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f2f682-aba8-43c6-b5bf-3474263893e8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:58.196345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 654/1315] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:58.201661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:58.209634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:58.210308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.212056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.212536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] PATH_INFO: `/v3/projects/673075fbfa05499fbb33fbda768ff128` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.217574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=673075fbfa05499fbb33fbda768ff128)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:58.220480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17534c6e-1e3b-4258-8473-3415e9acfb11 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:58.231983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:58.237506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dc4a4f-c144-4eb6-8c93-5fb49affecdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:58.238081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc4a4f-c144-4eb6-8c93-5fb49affecdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.238366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc4a4f-c144-4eb6-8c93-5fb49affecdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.238638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dc4a4f-c144-4eb6-8c93-5fb49affecdb admin admin] PATH_INFO: `/v3/projects/cd8a92c86f0845559187fcd2bbff07ea/users/75f64ef3503c42539533c0710055f1a7/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.250336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc4a4f-c144-4eb6-8c93-5fb49affecdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8a92c86f0845559187fcd2bbff07ea, user_id=75f64ef3503c42539533c0710055f1a7, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:58.253074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dc4a4f-c144-4eb6-8c93-5fb49affecdb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:58.292100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 655/1316] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:58 2022] PUT /identity/v3/projects/cd8a92c86f0845559187fcd2bbff07ea/users/75f64ef3503c42539533c0710055f1a7/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:30:58.345486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '673075fbfa05499fbb33fbda768ff128', 'request_id': 'req-1c500426-49fb-4e60-8658-562ec6c3c345'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:58.360592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '673075fbfa05499fbb33fbda768ff128', 'request_id': 'req-1c500426-49fb-4e60-8658-562ec6c3c345'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:58.372767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '673075fbfa05499fbb33fbda768ff128', 'request_id': 'req-1c500426-49fb-4e60-8658-562ec6c3c345'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:58.394729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] The token cache is being invalidate because project 673075fbfa05499fbb33fbda768ff128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:30:58.395737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:58.398103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1c500426-49fb-4e60-8658-562ec6c3c345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:58.404403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 662/1317] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:30:57 2022] DELETE /identity/v3/projects/673075fbfa05499fbb33fbda768ff128 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:30:58.406624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.406915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.407186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.446570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:58.448081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:58.478854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:58.571890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.572274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.572602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.575514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 656/1318] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:30:58.621945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:58.623571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:58.659056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 663/1319] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:30:58.663854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:58.683524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d72cea-f40d-4a9c-bc8b-dcf523ca223f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:58.698848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:58.699483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.699758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.700027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.700678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:30:58.701158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:58.702814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:58.740764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 657/1320] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:30:58.748067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:58.787182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6be8c9c-ba07-44ab-8d5c-14e1c4ac354c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:58.794096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:58.794972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:58.795493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:58.795862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:58.796658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:58.801752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:58.867492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 664/1321] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:58 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:58.873377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:58.913073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a7d37-4db4-43b3-95ad-715611889266 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:59.107422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:59.107794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.107998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.108246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.108892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:59.110846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:59.205146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 658/1322] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:30:58 2022] POST /identity/v3/users => generated 291 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:30:59.209378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:59.241619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f15d4-bb3a-40cd-8f16-cfcdadfb9dff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:59.250258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:59.253003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.274191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:59.274835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.275120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.275390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.276580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:59.278615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:59.286619 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4aba4-aad9-4745-8808-c09647ed165e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:59.290302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:59.307911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 665/1323] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:30:59.312540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:59.313710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:59.317968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.319557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.319943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] PATH_INFO: `/v3/users/fdae14d23d3e4eac8128fbefd4221be8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.339388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e3735-8dd2-4d40-bbe1-504522059bb0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:59.343130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdae14d23d3e4eac8128fbefd4221be8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:59.348686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:59.375821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:59.376543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.377044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.377412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.378271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:59.380080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:59.408269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 666/1324] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:30:59.412379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:59.435652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d062a705-d7a9-400c-9c19-dbee1b8913fe admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:30:59.445481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e3400-0640-41d5-ab90-95eed9eeee2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:30:59.456179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3400-0640-41d5-ab90-95eed9eeee2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.456629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3400-0640-41d5-ab90-95eed9eeee2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.457776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e3400-0640-41d5-ab90-95eed9eeee2f admin admin] PATH_INFO: `/v3/projects/da2684ea501046d19767ebf4d17630f6/users/fb641067fdb74737acbf50b1fe6e7ea9/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.489244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdae14d23d3e4eac8128fbefd4221be8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:59.490172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdae14d23d3e4eac8128fbefd4221be8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:59.492512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdae14d23d3e4eac8128fbefd4221be8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:59.512342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3400-0640-41d5-ab90-95eed9eeee2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=da2684ea501046d19767ebf4d17630f6, user_id=fb641067fdb74737acbf50b1fe6e7ea9, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:30:59.515267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75e3400-0640-41d5-ab90-95eed9eeee2f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:30:59.525450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdae14d23d3e4eac8128fbefd4221be8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:59.544670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 667/1325] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:30:59 2022] PUT /identity/v3/projects/da2684ea501046d19767ebf4d17630f6/users/fb641067fdb74737acbf50b1fe6e7ea9/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:30:59.560671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.561077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-baecde67-894d-4432-acd8-85b9e478c68b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdae14d23d3e4eac8128fbefd4221be8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:30:59.561581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.562068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.613698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 659/1326] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:30:59 2022] DELETE /identity/v3/users/fdae14d23d3e4eac8128fbefd4221be8 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:30:59.625820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:30:59.626176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:30:59.626375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:30:59.660870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] MFA Rules not processed for user `fb641067fdb74737acbf50b1fe6e7ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:59.675996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.706217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:30:59.731677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:30:59.739608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.968021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 668/1327] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:30:59.977678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:30:59.984907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:30:59.994190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 660/1328] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:00.017055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abc504-5d79-418f-80a1-0495cecdf52d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.023174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.024100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.025064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.025648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.027275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.043315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.046960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:00.048395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:00.148687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 669/1329] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:30:59 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:00.150237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:00.259131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df496cf6-3589-4f57-aa57-6c4fdffaa103 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.327945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af58710f-eb77-42a0-b150-3c266d239a3c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.335701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.336334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.336611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.336885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.337559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.340789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.375725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 670/1330] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:00 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:00.378980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:00.407918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae60cb9-8205-4b37-a622-083cb2785fba admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.414163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.415453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.417772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.419065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.424671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.426222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.427227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.428234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.431275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.435969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.448886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.454150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.474784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 671/1331] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:00.478328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:00.488656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 661/1332] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:00.499380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e1584-00e7-4087-9daa-2e46344d18b9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.507019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.508234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.508561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.509035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] PATH_INFO: `/v3/projects/2947df8b24c04bc5b86a08b6f56f97b0/users/a80fd9a75cc448d48e7adaeebafb31f9/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.523370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:00.538139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2947df8b24c04bc5b86a08b6f56f97b0, user_id=a80fd9a75cc448d48e7adaeebafb31f9, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.546061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.566769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 672/1333] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:00 2022] PUT /identity/v3/projects/2947df8b24c04bc5b86a08b6f56f97b0/users/a80fd9a75cc448d48e7adaeebafb31f9/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:00.571118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:00.587380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0c9135-d3af-4437-8d4c-a0436d314643 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.745237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99000e03-7dff-49fb-a47a-9814838a5b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.747306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000e03-7dff-49fb-a47a-9814838a5b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.747661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000e03-7dff-49fb-a47a-9814838a5b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.748034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99000e03-7dff-49fb-a47a-9814838a5b4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.748830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000e03-7dff-49fb-a47a-9814838a5b4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.751186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99000e03-7dff-49fb-a47a-9814838a5b4a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.789435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 662/1334] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:00.792937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:00.805837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14879a4-914f-4e17-aa65-c5ae7e3e0c57 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:00.813681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:00.814487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.814903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.815292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] PATH_INFO: `/v3/projects/2947df8b24c04bc5b86a08b6f56f97b0/users/a80fd9a75cc448d48e7adaeebafb31f9/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.820538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=2947df8b24c04bc5b86a08b6f56f97b0, user_id=a80fd9a75cc448d48e7adaeebafb31f9, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:00.825368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:00.842358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 673/1335] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:00 2022] PUT /identity/v3/projects/2947df8b24c04bc5b86a08b6f56f97b0/users/a80fd9a75cc448d48e7adaeebafb31f9/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:31:00.854840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:00.855150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:00.855516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:00.871554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:00.913739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] MFA Rules not processed for user `a80fd9a75cc448d48e7adaeebafb31f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:00.915082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:01.021816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 663/1336] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:01.035471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:01.038435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:01.142325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb173b-7549-4846-b9d2-3701812384bf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:01.149488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9916f-f8ef-4879-9d7c-41de37df7def service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:01.150588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9916f-f8ef-4879-9d7c-41de37df7def service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:01.151704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9916f-f8ef-4879-9d7c-41de37df7def service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:01.152568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9916f-f8ef-4879-9d7c-41de37df7def service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:01.173354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9916f-f8ef-4879-9d7c-41de37df7def service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:01.176759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9916f-f8ef-4879-9d7c-41de37df7def service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:01.200554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 674/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:01.420130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:01.433996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c731e81-9ef2-47a8-9807-dbf481ede923 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:01.444989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:01.446191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:01.446579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:01.446952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:01.606207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:01.609038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:01.634193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 664/1338] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:02.718056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:02.718857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26f90-3bf7-4046-a1be-ab47fdc62a5c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:02.719250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26f90-3bf7-4046-a1be-ab47fdc62a5c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:02.719414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c26f90-3bf7-4046-a1be-ab47fdc62a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:02.722713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:02.844514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-05c26f90-3bf7-4046-a1be-ab47fdc62a5c None None] MFA Rules not processed for user `75f64ef3503c42539533c0710055f1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:02.847971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-05c26f90-3bf7-4046-a1be-ab47fdc62a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:02.923892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5afdf17-343a-4a8e-94ec-e2b24d3d1c3d service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:02.929275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:02.929960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:02.930261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:02.930523 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] PATH_INFO: `/v3/projects/0798aa39fc88488882c81a36c176d5ff` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:02.931814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0798aa39fc88488882c81a36c176d5ff)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:02.932988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: WARNING py.warnings [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:31:02.932988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: warnings.warn(msg) Oct 12 12:31:02.932988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: Oct 12 12:31:02.934610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:03.027328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:03.079583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 675/1339] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:03.090001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0798aa39fc88488882c81a36c176d5ff', 'request_id': 'req-8d1cf1b1-8b07-476f-96a3-165847b681ef'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:03.107574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0798aa39fc88488882c81a36c176d5ff', 'request_id': 'req-8d1cf1b1-8b07-476f-96a3-165847b681ef'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:03.134421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0798aa39fc88488882c81a36c176d5ff', 'request_id': 'req-8d1cf1b1-8b07-476f-96a3-165847b681ef'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:03.165002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] The token cache is being invalidate because project 0798aa39fc88488882c81a36c176d5ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:03.165002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:03.165002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8d1cf1b1-8b07-476f-96a3-165847b681ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:03.168705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 665/1340] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:02 2022] DELETE /identity/v3/projects/0798aa39fc88488882c81a36c176d5ff => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:31:03.173386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:03.173708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:03.174016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:03.232402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:03.235846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:03.458042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 676/1341] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:03.473961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:03.473961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:03.473961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:03.516119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:03.517407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:03.561984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 666/1342] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:03.566753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:03.574161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:03.589597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a3da1-b2bc-4b66-8822-1e4f84414ca6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:03.620284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:03.620760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:03.621116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:03.628100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:03.628945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:03.631422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:03.638694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:03.722925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 677/1343] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:03.727335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:03.743862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4fac4-fea3-4a2e-8fda-ae56fff3d215 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:03.770054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5e150-3238-4a03-960b-ae6e3023df71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:03.770973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5e150-3238-4a03-960b-ae6e3023df71 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:03.771259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5e150-3238-4a03-960b-ae6e3023df71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:03.771542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5e150-3238-4a03-960b-ae6e3023df71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:03.772287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5e150-3238-4a03-960b-ae6e3023df71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:03.779972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5e150-3238-4a03-960b-ae6e3023df71 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:03.871527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 667/1344] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:03 2022] POST /identity/v3/projects => generated 365 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:03.875824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:03.889964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaa7663-c790-4706-99fb-b0f2a5d70be0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.003600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a8351-bcf9-4cb9-a0f5-7d2b70a6c46f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.003950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a8351-bcf9-4cb9-a0f5-7d2b70a6c46f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.004510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a8351-bcf9-4cb9-a0f5-7d2b70a6c46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.101909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c47a8351-bcf9-4cb9-a0f5-7d2b70a6c46f None None] MFA Rules not processed for user `76e3059b01884215becf97694c3b7027`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:04.109520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c47a8351-bcf9-4cb9-a0f5-7d2b70a6c46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:04.115474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:04.116814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.117884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.119078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.120678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:04.127011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:04.177900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 678/1345] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:03 2022] POST /identity/v3/users => generated 283 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:04.178823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.199718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df3cc67-af04-42ef-b8fb-3aadc977d485 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.220382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:04.221105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.223131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.223260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.224930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:04.228421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:04.264230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 679/1346] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:04.270308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.290687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 668/1347] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:04.299789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d34318-9879-4e75-bb0d-4f18b680d3d3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.302878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.303076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.303347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.311199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:04.311976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.313497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.313779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.314458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:04.318483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:04.349079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 680/1348] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:04.355701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.358126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] MFA Rules not processed for user `76e3059b01884215becf97694c3b7027`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:04.359396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:04.379585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 669/1349] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:04.380062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fcc429-c5da-46ae-85b3-89067688baf0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.392043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.393752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:04.399798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f2c71-e6ca-4fd2-b159-ea66e80b8378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:04.407268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f2c71-e6ca-4fd2-b159-ea66e80b8378 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.407268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f2c71-e6ca-4fd2-b159-ea66e80b8378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.407268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f2c71-e6ca-4fd2-b159-ea66e80b8378 admin admin] PATH_INFO: `/v3/projects/1058274c4f6d4b53bd8e15c901c509f7/users/aa3dfe2f60f54add8d94944ef7d732a8/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.437558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f2c71-e6ca-4fd2-b159-ea66e80b8378 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1058274c4f6d4b53bd8e15c901c509f7, user_id=aa3dfe2f60f54add8d94944ef7d732a8, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:04.444957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f2c71-e6ca-4fd2-b159-ea66e80b8378 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:04.484562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 681/1350] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:04 2022] PUT /identity/v3/projects/1058274c4f6d4b53bd8e15c901c509f7/users/aa3dfe2f60f54add8d94944ef7d732a8/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:04.505007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.505288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.505686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.561184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] MFA Rules not processed for user `aa3dfe2f60f54add8d94944ef7d732a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:04.564326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:04.590641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c8d5a-93fc-4e3b-a7a6-ca17365e6f4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.603416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:04.701413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 682/1351] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:04.705230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.706103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:04.707377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.707829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.708271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.735572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:04.806641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc6b7c-2dae-47da-a012-a2c8c632897f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:04.814143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:04.814908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:04.815705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:04.816092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:04.817299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:04.820744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:04.839345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:04.841467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:04.856511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 670/1352] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:04.877440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 683/1353] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:04 2022] POST /identity/v3/projects => generated 365 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:04.882655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:04.900589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96979127-faca-4e04-8d87-cd00708e4e23 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:05.079011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:05.079725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.080010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.080284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.080935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:05.085120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:05.137875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 671/1354] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:04 2022] POST /identity/v3/users => generated 289 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:05.142330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:05.160588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abc1cf-9148-43b3-b13a-2c34e8841bbd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:05.172557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:05.173389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.173784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.174160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.174921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:05.176755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:05.206058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 684/1355] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:05.210321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:05.226586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f2ff4e-1167-4e87-9847-97ddbdf802b2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:05.240360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:05.241164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.241573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.243034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] PATH_INFO: `/v3/projects/419203e8fefa4cba9f2579f5fa821f46/users/dbfb2b5517624056ba5ef51f91a50823/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.282110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] RBAC: Authorizing `identity:create_grant(project_id=419203e8fefa4cba9f2579f5fa821f46, user_id=dbfb2b5517624056ba5ef51f91a50823, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:05.294481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:05.331455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 672/1356] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:05 2022] PUT /identity/v3/projects/419203e8fefa4cba9f2579f5fa821f46/users/dbfb2b5517624056ba5ef51f91a50823/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:05.335526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:05.349803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0d3e9-23b4-4a7c-82b4-d14a9a9b2fbb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:05.461057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa075c-1017-4400-a0ca-2439fa5fc816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:05.462228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa075c-1017-4400-a0ca-2439fa5fc816 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.462625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa075c-1017-4400-a0ca-2439fa5fc816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.462998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa075c-1017-4400-a0ca-2439fa5fc816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.464141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa075c-1017-4400-a0ca-2439fa5fc816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:05.466564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa075c-1017-4400-a0ca-2439fa5fc816 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:05.485121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 685/1357] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:05.489907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:05.524001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bf0333-3ebc-43d6-a741-7f86588bd82b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:05.554246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:05.555091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.555730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.556155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] PATH_INFO: `/v3/projects/419203e8fefa4cba9f2579f5fa821f46/users/dbfb2b5517624056ba5ef51f91a50823/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.559933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=419203e8fefa4cba9f2579f5fa821f46, user_id=dbfb2b5517624056ba5ef51f91a50823, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:05.563925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:05.574372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:05.591903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 673/1358] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:05 2022] PUT /identity/v3/projects/419203e8fefa4cba9f2579f5fa821f46/users/dbfb2b5517624056ba5ef51f91a50823/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:05.602967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:05.603469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:05.603831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:05.677158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] MFA Rules not processed for user `dbfb2b5517624056ba5ef51f91a50823`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:05.678555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:05.869356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 686/1359] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:05.881239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:05.884541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:06.053112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0080d-f9b1-4176-b2c9-aa93810d5ec4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:06.060230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0c77f-7254-4a0b-95b4-45c88e1e34e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:06.060929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c77f-7254-4a0b-95b4-45c88e1e34e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:06.061281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c77f-7254-4a0b-95b4-45c88e1e34e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:06.061635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0c77f-7254-4a0b-95b4-45c88e1e34e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:06.084458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0c77f-7254-4a0b-95b4-45c88e1e34e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:06.086223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0c77f-7254-4a0b-95b4-45c88e1e34e2 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:06.110707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 674/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:06.288683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:06.290911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:06.320784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ac32a-de6d-40f1-be0a-f032369b7eca None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:06.330182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76ded9-c2b8-4579-8737-d1afbd1f195c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:06.330984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ded9-c2b8-4579-8737-d1afbd1f195c service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:06.331286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ded9-c2b8-4579-8737-d1afbd1f195c service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:06.331565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76ded9-c2b8-4579-8737-d1afbd1f195c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:06.372384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:06.503769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76ded9-c2b8-4579-8737-d1afbd1f195c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:06.506648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76ded9-c2b8-4579-8737-d1afbd1f195c service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:06.533904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 687/1361] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:07.200769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.201650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.202110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.270627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] MFA Rules not processed for user `25119286009743bbac4217d5faa40177`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:07.272272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:07.424923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 675/1362] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:07.425288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fee1ae-52b8-4f5f-ac32-6273ffde1105 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.425896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fee1ae-52b8-4f5f-ac32-6273ffde1105 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.425896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fee1ae-52b8-4f5f-ac32-6273ffde1105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.477144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-83fee1ae-52b8-4f5f-ac32-6273ffde1105 None None] MFA Rules not processed for user `25119286009743bbac4217d5faa40177`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:07.482071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-83fee1ae-52b8-4f5f-ac32-6273ffde1105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:07.514351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 688/1363] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:07.534794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:07.538717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:07.670772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683444f-4c7f-4fec-939a-f9a9b404db5b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:07.690338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:07.691308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.691808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.692476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.703102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:07.706569 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:07.706733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910939-0bf9-484f-a905-bbd96e7b09ca None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.706974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910939-0bf9-484f-a905-bbd96e7b09ca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.707280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910939-0bf9-484f-a905-bbd96e7b09ca None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.709830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 689/1364] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:31:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:07.737251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf47a0e-d258-4703-be1a-f2c0bd777979 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:07.737848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf47a0e-d258-4703-be1a-f2c0bd777979 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:07.740416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf47a0e-d258-4703-be1a-f2c0bd777979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:07.743089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 676/1365] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:07.834706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-eaf47a0e-d258-4703-be1a-f2c0bd777979 None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:07.842524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eaf47a0e-d258-4703-be1a-f2c0bd777979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.005941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 690/1366] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Oct 12 12:31:08.394333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:08.396553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.436248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b8c45-a92f-44bd-aeb8-64126152c9ff service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:08.453842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933e8f9-0f5e-4eed-9997-05da657025bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:08.454474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933e8f9-0f5e-4eed-9997-05da657025bb service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.454762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933e8f9-0f5e-4eed-9997-05da657025bb service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.455038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933e8f9-0f5e-4eed-9997-05da657025bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.480807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933e8f9-0f5e-4eed-9997-05da657025bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:08.482516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933e8f9-0f5e-4eed-9997-05da657025bb service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:08.507838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 677/1367] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:08.661217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349d1fb-0c03-4f31-9709-f9548a61d9d3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.663150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349d1fb-0c03-4f31-9709-f9548a61d9d3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.663662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b349d1fb-0c03-4f31-9709-f9548a61d9d3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.666140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 691/1368] 10.222.0.17 () {54 vars in 912 bytes} [Wed Oct 12 12:31:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:08.673043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.673431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.673719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.686631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:08.744669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:08.749301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.813992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0a1aa-f8d8-4a06-abf3-2eacdcae854a None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.814271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0a1aa-f8d8-4a06-abf3-2eacdcae854a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.814552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f0a1aa-f8d8-4a06-abf3-2eacdcae854a None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.816061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 692/1369] 10.222.0.17 () {54 vars in 912 bytes} [Wed Oct 12 12:31:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:08.824242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.824527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.824809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.842661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 678/1370] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:08.852711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:08.854717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.866367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:08.868647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.889204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 693/1371] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:08.901549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63b0074-df09-4263-8aeb-9d42cf04d186 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:08.910906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:08.912747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.913037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.913331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.935267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:08.937359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:08.938592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:08.942322 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:08.952173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 679/1372] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:08.959685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35690710-22c3-49cf-90f0-9aff814aeab2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:08.971268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008d1d8-c25d-4b34-a265-e9d228b79351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:08.972232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008d1d8-c25d-4b34-a265-e9d228b79351 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:08.972676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008d1d8-c25d-4b34-a265-e9d228b79351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:08.973126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008d1d8-c25d-4b34-a265-e9d228b79351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:08.992396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008d1d8-c25d-4b34-a265-e9d228b79351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:08.995703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008d1d8-c25d-4b34-a265-e9d228b79351 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:09.018711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 694/1373] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:10.190181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:10.207261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a5e9fd-cca3-4da2-8aac-dbb1de8dee15 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:10.213910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:10.214973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.215478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.215974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.218134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:10.227796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2f74-ee94-4883-891f-0282857550b3 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.228641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2f74-ee94-4883-891f-0282857550b3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.229281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc2f74-ee94-4883-891f-0282857550b3 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.233911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 695/1374] 10.222.0.17 () {54 vars in 961 bytes} [Wed Oct 12 12:31:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:10.239827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4761-b1cc-4a66-9136-83a7cf6247fb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.240447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4761-b1cc-4a66-9136-83a7cf6247fb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.240944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86d4761-b1cc-4a66-9136-83a7cf6247fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.254827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:10.256808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:10.278283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a86d4761-b1cc-4a66-9136-83a7cf6247fb None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:10.283066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a86d4761-b1cc-4a66-9136-83a7cf6247fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:10.291186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 680/1375] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:10.308818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 696/1376] 10.222.0.17 () {58 vars in 1058 bytes} [Wed Oct 12 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:10.319470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:10.341919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63bf094-87aa-4e17-aec1-c3acdfe864c1 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:10.348321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509e8c7-11c3-489a-8c32-5b9a998ab63a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:10.349133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e8c7-11c3-489a-8c32-5b9a998ab63a service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.349805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e8c7-11c3-489a-8c32-5b9a998ab63a service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.350253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509e8c7-11c3-489a-8c32-5b9a998ab63a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.387340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15377853-3c0e-4676-92eb-18d00bd67a25 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.388464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15377853-3c0e-4676-92eb-18d00bd67a25 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.388964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15377853-3c0e-4676-92eb-18d00bd67a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.408701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e8c7-11c3-489a-8c32-5b9a998ab63a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:10.410461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509e8c7-11c3-489a-8c32-5b9a998ab63a service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:10.424994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-15377853-3c0e-4676-92eb-18d00bd67a25 None None] MFA Rules not processed for user `76e3059b01884215becf97694c3b7027`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:10.426495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-15377853-3c0e-4676-92eb-18d00bd67a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:10.434505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 681/1377] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:10.641961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 697/1378] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:10.648383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcdaa1-6a42-408a-906a-5d79b8be8127 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.648669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcdaa1-6a42-408a-906a-5d79b8be8127 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.648992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfcdaa1-6a42-408a-906a-5d79b8be8127 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.650526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 682/1379] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:31:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:10.669828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.670110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.670387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.707139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] MFA Rules not processed for user `8740a4be0475453fb422383b9b335d86`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:10.708362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:10.724406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 698/1380] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:10.767454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcdaa1-6a42-408a-906a-5d79b8be8127 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:10.805871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfcdaa1-6a42-408a-906a-5d79b8be8127 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:10.841270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:10.842691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:10.846006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:10.846630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.846907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.847175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.879077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:10.886881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:10.890457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:10.922321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 683/1381] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:10.970350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b953faf-1ce4-41d6-9e6b-a4f8a7e6d2fe None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:10.980647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:10.987840 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:10.988217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:10.988555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] PATH_INFO: `/v3/users/3391253ce97247c5a45ddcbdcc06e935` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:10.990245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3391253ce97247c5a45ddcbdcc06e935)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:10.991935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: WARNING py.warnings [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 12 12:31:10.991935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: warnings.warn(msg) Oct 12 12:31:10.991935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: Oct 12 12:31:10.993952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:11.139562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3391253ce97247c5a45ddcbdcc06e935'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:11.160455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3391253ce97247c5a45ddcbdcc06e935'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:11.162000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3391253ce97247c5a45ddcbdcc06e935'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:11.210961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3391253ce97247c5a45ddcbdcc06e935'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:11.211597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3391253ce97247c5a45ddcbdcc06e935'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:11.422272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 699/1382] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:10 2022] DELETE /identity/v3/users/3391253ce97247c5a45ddcbdcc06e935 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:12.189557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:12.218223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c73db5-cf21-4dd3-9440-209fcdfb2105 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:12.491693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:12.498898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:12.499399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:12.499810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] PATH_INFO: `/v3/projects/fe8cfe97aa2d474d8c78821b297c6c80` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:12.501262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8cfe97aa2d474d8c78821b297c6c80)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:12.514087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:12.645030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:12.647399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8cfe97aa2d474d8c78821b297c6c80', 'request_id': 'req-1f2629c1-1fcf-4126-80fc-318d5f138a6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:12.666228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8cfe97aa2d474d8c78821b297c6c80', 'request_id': 'req-1f2629c1-1fcf-4126-80fc-318d5f138a6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:12.691019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8cfe97aa2d474d8c78821b297c6c80', 'request_id': 'req-1f2629c1-1fcf-4126-80fc-318d5f138a6b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:12.716370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] The token cache is being invalidate because project fe8cfe97aa2d474d8c78821b297c6c80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:12.716669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:12.717369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1f2629c1-1fcf-4126-80fc-318d5f138a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:12.719015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 684/1383] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:12 2022] DELETE /identity/v3/projects/fe8cfe97aa2d474d8c78821b297c6c80 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:12.723042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:12.724609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:12.765488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faf3b8-dd3e-45b6-b36b-a9c370a6d00e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:12.861680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:12.862206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:12.862661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:12.918000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] MFA Rules not processed for user `abbc8a34ac8f44328596102c35e184dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:12.919793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:12.952881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:12.953848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:12.954270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:12.954683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] PATH_INFO: `/v3/users/75f64ef3503c42539533c0710055f1a7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:12.958080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=75f64ef3503c42539533c0710055f1a7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:12.960286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:13.059035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 685/1384] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:13.069350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:13.086284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:13.091935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.098304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75f64ef3503c42539533c0710055f1a7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.116847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4cb44-b720-4ac4-b317-e858f887dffc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:13.122365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:13.123011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.123331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.123614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] PATH_INFO: `/v3/users/594363cac9524600a7be433905ee3d0c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.124785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=594363cac9524600a7be433905ee3d0c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:13.126812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:13.136877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75f64ef3503c42539533c0710055f1a7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.140211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75f64ef3503c42539533c0710055f1a7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.158850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75f64ef3503c42539533c0710055f1a7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.160065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-00fa7bd2-8c73-4f14-b001-200cb0c88f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75f64ef3503c42539533c0710055f1a7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.221631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '594363cac9524600a7be433905ee3d0c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.224691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '594363cac9524600a7be433905ee3d0c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.225736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '594363cac9524600a7be433905ee3d0c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.234635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '594363cac9524600a7be433905ee3d0c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.238064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 700/1385] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:12 2022] DELETE /identity/v3/users/75f64ef3503c42539533c0710055f1a7 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:13.251595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '594363cac9524600a7be433905ee3d0c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:13.290464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 686/1386] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:13 2022] DELETE /identity/v3/users/594363cac9524600a7be433905ee3d0c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:31:13.295332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.295620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.295896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.302640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:13.305555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.321099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:13.322435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.392642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:13.412565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 701/1387] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:13.454827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:13.457959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.477591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15265add-f969-4945-8e2e-d73727d24d2c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:13.491330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:13.493063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.493696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.493902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.496602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.651731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:13.658035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc97334f-597d-4eb7-bb69-9ec7cafc9109 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:13.685953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 687/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:13.687228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.687519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.687794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.797338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] MFA Rules not processed for user `383004527ce84e4f9fc40760a040936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:13.800307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.809067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82dc015-13d7-4e4d-9ee0-7c6e1d64d95c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:13.826783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:13.827470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.828071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.828349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.874220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:13.882300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:13.921689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 702/1389] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:13.922706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:13.926627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:13.968595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397a46a-2f96-4cf0-986a-e08ae2a66afc service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:13.984260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:13.991478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:13.991766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:13.992205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] PATH_INFO: `/v3/projects/cd8a92c86f0845559187fcd2bbff07ea` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:13.997430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8a92c86f0845559187fcd2bbff07ea)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:14.027832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:14.125588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 688/1390] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:14.125588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:14.126901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.211251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3848c-1f8f-4df3-9c70-f1530d407530 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:14.258319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8a92c86f0845559187fcd2bbff07ea', 'request_id': 'req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.277581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8a92c86f0845559187fcd2bbff07ea', 'request_id': 'req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.282745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:14.309173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8a92c86f0845559187fcd2bbff07ea', 'request_id': 'req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.325043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:14.325925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:14.326133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:14.326407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] PATH_INFO: `/v3/users/aa3dfe2f60f54add8d94944ef7d732a8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:14.331216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] The token cache is being invalidate because project cd8a92c86f0845559187fcd2bbff07ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:14.333084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.333543 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3dfe2f60f54add8d94944ef7d732a8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:14.335872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.339895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:14.340743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 703/1391] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:13 2022] DELETE /identity/v3/projects/cd8a92c86f0845559187fcd2bbff07ea => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:14.342318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:14.346569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.352199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:14.507176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3dfe2f60f54add8d94944ef7d732a8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.512187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3dfe2f60f54add8d94944ef7d732a8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.513288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3dfe2f60f54add8d94944ef7d732a8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.539108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3dfe2f60f54add8d94944ef7d732a8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.564812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-367439ae-7a28-4f3d-ab15-b674f58f608b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3dfe2f60f54add8d94944ef7d732a8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:14.618898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 689/1392] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:14 2022] DELETE /identity/v3/users/aa3dfe2f60f54add8d94944ef7d732a8 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:14.623907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213432e4-e24a-4bfe-bf58-0f0067181bbd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:14.629670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213432e4-e24a-4bfe-bf58-0f0067181bbd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:14.630340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213432e4-e24a-4bfe-bf58-0f0067181bbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:14.637576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e40f9-3c32-4d38-8c0e-4e9c7636f107 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:14.675758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-213432e4-e24a-4bfe-bf58-0f0067181bbd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:14.679649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-213432e4-e24a-4bfe-bf58-0f0067181bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.863127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 690/1393] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:14.866239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d2e4-df6c-44ae-aa75-75fb9c813cf2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:14.867733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d2e4-df6c-44ae-aa75-75fb9c813cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:14.868045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8d2e4-df6c-44ae-aa75-75fb9c813cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:14.921804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-69a8d2e4-df6c-44ae-aa75-75fb9c813cf2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:14.924540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-69a8d2e4-df6c-44ae-aa75-75fb9c813cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.936664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:14.938127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:14.938609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:14.950681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:14.962433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 691/1394] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:14.962915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:14.964623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b2a01-d8c9-4b1b-ad2f-46338aac0004 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:14.964903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b2a01-d8c9-4b1b-ad2f-46338aac0004 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:14.965543 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21b2a01-d8c9-4b1b-ad2f-46338aac0004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:14.996147 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c21b2a01-d8c9-4b1b-ad2f-46338aac0004 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:14.997931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c21b2a01-d8c9-4b1b-ad2f-46338aac0004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.016633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 692/1395] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:15.019978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.020268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.020546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.122886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:15.125278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:15.151769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 704/1396] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 810 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:15.154417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.178300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21514ad-e18d-4d7a-95af-5f54668152e1 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:15.192026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:15.193201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.193901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.194386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.195647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:15.196768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:15.200423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:15.223786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 705/1397] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:15.225526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.226324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:15.227527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.228751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.251051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:15.419860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:15.470301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 693/1398] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:15.470301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.496253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45455748-8b22-46d6-a7c8-ec24c14722de None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:15.529284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:15.529963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.530246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.530521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.531186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:15.533142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:15.547624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfefe7e-a03b-4110-bd23-d3c9d8abb6e3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:15.560129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:15.562480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.562905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.563366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.586081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:15.589161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:15.613622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 706/1399] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:15.614293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 694/1400] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:15 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:15.615794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.634542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd45375-fa83-406c-80ca-9203d0c9bb43 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:15.773949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.777664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:15.787818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:15.789217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.789522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.789798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.792713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:15.795465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:15.862653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 707/1401] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:15 2022] POST /identity/v3/users => generated 282 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:15.864541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.879926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27deaffc-a46a-4cc3-b256-262377d2f307 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:15.890094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:15.891723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.893961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.894184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.896638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:15.904234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:15.941219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 708/1402] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:15.949853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:15.971522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713c4ee-612e-4315-b169-4c394e40bf85 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:15.991909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:15.992855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:15.993583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:15.994169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:15.998531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f36a10f-8995-42da-b2a0-28c5d65e02ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:16.000764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:16.003544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:16.008341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:16.009032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.010850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.011146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:16.034447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 709/1403] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:31:16.040285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:16.091229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03830d-8447-4d43-9cf6-936dd1877fba admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:16.111579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e5bf5-5fec-4f70-8aea-d3b0d5fa2a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:16.113837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5bf5-5fec-4f70-8aea-d3b0d5fa2a8f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.115600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5bf5-5fec-4f70-8aea-d3b0d5fa2a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.116363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e5bf5-5fec-4f70-8aea-d3b0d5fa2a8f admin admin] PATH_INFO: `/v3/projects/4bfc5899b0874582b1d4721c7741bc8d/users/2fbf56b734a443ccb2dee9ab463e1d00/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:16.148025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5bf5-5fec-4f70-8aea-d3b0d5fa2a8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bfc5899b0874582b1d4721c7741bc8d, user_id=2fbf56b734a443ccb2dee9ab463e1d00, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:16.158722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e5bf5-5fec-4f70-8aea-d3b0d5fa2a8f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:16.245974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 710/1404] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:16 2022] PUT /identity/v3/projects/4bfc5899b0874582b1d4721c7741bc8d/users/2fbf56b734a443ccb2dee9ab463e1d00/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:16.261275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.269898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.270397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:16.374609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:16.376577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:16.378388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] MFA Rules not processed for user `2fbf56b734a443ccb2dee9ab463e1d00`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:16.379790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:16.401904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 695/1405] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:31:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:31:16.418065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:16.430146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:16.501066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd67970d-b042-4d31-9d8b-c5f3c377e7e1 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:16.517345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:16.654486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 711/1406] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:16.659756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:16.670656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:16.729939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:16.731176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:16.732744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:16.733400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] PATH_INFO: `/v3/projects/3c899a3604654d39b0b25659c83a3c25` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:16.737347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c899a3604654d39b0b25659c83a3c25)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:16.743115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:16.909257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c899a3604654d39b0b25659c83a3c25', 'request_id': 'req-4552decc-b56e-4986-8cb9-977caa9e74dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:16.935649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c899a3604654d39b0b25659c83a3c25', 'request_id': 'req-4552decc-b56e-4986-8cb9-977caa9e74dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:16.955189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c899a3604654d39b0b25659c83a3c25', 'request_id': 'req-4552decc-b56e-4986-8cb9-977caa9e74dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:16.965429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c4f798-fe25-45c5-a2e0-24e1cb517c2f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:16.996259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] The token cache is being invalidate because project 3c899a3604654d39b0b25659c83a3c25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:16.996576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:17.000833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4552decc-b56e-4986-8cb9-977caa9e74dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:17.009444 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 696/1407] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:16 2022] DELETE /identity/v3/projects/3c899a3604654d39b0b25659c83a3c25 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:17.016345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84cc2-a065-46d8-b13b-62f5bc6600e6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.016658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84cc2-a065-46d8-b13b-62f5bc6600e6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.016945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a84cc2-a065-46d8-b13b-62f5bc6600e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.088580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-39a84cc2-a065-46d8-b13b-62f5bc6600e6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:17.093555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-39a84cc2-a065-46d8-b13b-62f5bc6600e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.194090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.198616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.199044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.199252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.204786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.211708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.319538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 697/1408] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:17.321846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.322144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.326403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.353707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 712/1409] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:16 2022] POST /identity/v3/projects => generated 363 bytes in 694 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:17.359396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.369999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.403365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:17.404642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:17.616893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 698/1410] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:17.618482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.639229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0d8221-e030-49d0-8e9a-17f2b5aeb58a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:17.641187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b413161-7a2b-4ad1-a9ba-b57e3a38ddf3 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:17.651665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.653272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.653789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.654077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.654352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.655466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.655743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.656247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.656950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.658577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:17.658890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.660510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.662937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.685467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:17.701824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 699/1411] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:17.702995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.726831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 713/1412] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/users => generated 288 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:17.733752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.741512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a1091f-f8da-411d-aa2f-52c5b4f491bf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:17.747325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:17.762443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.763745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.764016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.764278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.765200 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdefe2c-5750-4b8c-8ead-75579c2a8739 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:17.766983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.771926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.776014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:17.777446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:17.777744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:17.778013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:17.781517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:17.786337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:17.819103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 714/1413] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:17.819953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.851835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 700/1414] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:17.855827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:17.856197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d0341-e1a3-4be3-90d6-bbcee9f7ede4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.022683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e8ffc-4ca1-4f7b-8a34-c23491c8d6d6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.032031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.033230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.033526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.034203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.036110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.043361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.051298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.051298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.051298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.051298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] PATH_INFO: `/v3/projects/c1065b83822f4207b293673e8ba99941/users/36c436a9a6d945158df1efaea3058bf1/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.089011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 701/1415] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:17 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:18.089011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:18.099614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1065b83822f4207b293673e8ba99941, user_id=36c436a9a6d945158df1efaea3058bf1, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.103498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.104560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae16132-d061-4d9c-94c5-702982284b50 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.111171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.111799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.112082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.112357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.113024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.115545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.138741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 702/1416] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:18.140316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:18.144151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 715/1417] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:17 2022] PUT /identity/v3/projects/c1065b83822f4207b293673e8ba99941/users/36c436a9a6d945158df1efaea3058bf1/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:18.148623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:18.156955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf48adf-c31b-4d25-b6ef-0a85aeec8930 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.304455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.307451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.308106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.309767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.311495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.314660 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.341823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 703/1418] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:18.351816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:18.397386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44231a-2e1a-46d4-a5e7-9b599076b4ed admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.417278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.419165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.420376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.420376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] PATH_INFO: `/v3/projects/92938ce5f9434f6796e067d05ca6a2e9/users/95625040769c4a4582ae52117ee1a1fd/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.428396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045298cc-92dd-4f4b-84a0-cf04bc90184d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.455212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.455861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.457296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.457615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.460466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.477302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.517075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92938ce5f9434f6796e067d05ca6a2e9, user_id=95625040769c4a4582ae52117ee1a1fd, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.523583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.528226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 716/1419] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:18 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:18.530701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:18.541241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:18.583081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 704/1420] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:18 2022] PUT /identity/v3/projects/92938ce5f9434f6796e067d05ca6a2e9/users/95625040769c4a4582ae52117ee1a1fd/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:18.584987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:18.604318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be62eec-6726-4c60-81fc-c3986206cf5f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.633354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:18.749874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433f9f28-0dd1-4c2d-890b-94ddc1b1b9b3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:18.772113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c565d59b-ba5e-4a77-9e8c-953712af2e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.772959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565d59b-ba5e-4a77-9e8c-953712af2e3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.773456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565d59b-ba5e-4a77-9e8c-953712af2e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.773783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c565d59b-ba5e-4a77-9e8c-953712af2e3a admin admin] PATH_INFO: `/v3/projects/c1065b83822f4207b293673e8ba99941/users/36c436a9a6d945158df1efaea3058bf1/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.778142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565d59b-ba5e-4a77-9e8c-953712af2e3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1065b83822f4207b293673e8ba99941, user_id=36c436a9a6d945158df1efaea3058bf1, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.780960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c565d59b-ba5e-4a77-9e8c-953712af2e3a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.821663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 705/1421] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:18 2022] PUT /identity/v3/projects/c1065b83822f4207b293673e8ba99941/users/36c436a9a6d945158df1efaea3058bf1/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:18.823880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:18.824536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.824817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.825083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] PATH_INFO: `/v3/projects/1058274c4f6d4b53bd8e15c901c509f7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.831927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1058274c4f6d4b53bd8e15c901c509f7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:18.833933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:18.834376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b6686-ae0d-4189-88b1-82a4d2d79307 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:18.834857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b6686-ae0d-4189-88b1-82a4d2d79307 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:18.835319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b6686-ae0d-4189-88b1-82a4d2d79307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:18.886657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-086b6686-ae0d-4189-88b1-82a4d2d79307 None None] MFA Rules not processed for user `95625040769c4a4582ae52117ee1a1fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:18.888185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-086b6686-ae0d-4189-88b1-82a4d2d79307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:18.890828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:18.980613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1058274c4f6d4b53bd8e15c901c509f7', 'request_id': 'req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.001115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1058274c4f6d4b53bd8e15c901c509f7', 'request_id': 'req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.022754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1058274c4f6d4b53bd8e15c901c509f7', 'request_id': 'req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.056403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] The token cache is being invalidate because project 1058274c4f6d4b53bd8e15c901c509f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:19.059064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.063030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.076883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 717/1422] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:18 2022] DELETE /identity/v3/projects/1058274c4f6d4b53bd8e15c901c509f7 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:19.078134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:19.079872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:19.113879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 706/1423] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:19.140865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:19.141241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:19.141723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:19.195371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] MFA Rules not processed for user `36c436a9a6d945158df1efaea3058bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:19.196748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:19.317834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed9c9b-b266-43a7-9523-2a4d2a50c04f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:19.332225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:19.332898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:19.333186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:19.333488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] PATH_INFO: `/v3/users/fb641067fdb74737acbf50b1fe6e7ea9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:19.339752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb641067fdb74737acbf50b1fe6e7ea9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:19.343108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:19.383183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 707/1424] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:19.388027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:19.392924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:19.469712 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7281002-7923-4064-b334-4c749bad97f5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:19.492201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:19.493844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:19.496660 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:19.497146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] PATH_INFO: `/v3/users/dbfb2b5517624056ba5ef51f91a50823` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:19.502202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbfb2b5517624056ba5ef51f91a50823)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:19.508160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:19.545960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb641067fdb74737acbf50b1fe6e7ea9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.577200 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb641067fdb74737acbf50b1fe6e7ea9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.579936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb641067fdb74737acbf50b1fe6e7ea9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.614030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:19.640116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb641067fdb74737acbf50b1fe6e7ea9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.643117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb641067fdb74737acbf50b1fe6e7ea9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.644857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbfb2b5517624056ba5ef51f91a50823'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.646214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbfb2b5517624056ba5ef51f91a50823'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.647220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbfb2b5517624056ba5ef51f91a50823'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.662571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbfb2b5517624056ba5ef51f91a50823'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.678528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbfb2b5517624056ba5ef51f91a50823'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:19.714248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 708/1425] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:19 2022] DELETE /identity/v3/users/dbfb2b5517624056ba5ef51f91a50823 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:19.715442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:19.717913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:19.738231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 718/1426] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:19 2022] DELETE /identity/v3/users/fb641067fdb74737acbf50b1fe6e7ea9 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:19.741178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:19.746007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:19.810865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:19.856970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0735a47-e952-4f0b-b495-dd428b5cd73b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:19.863257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb150f7-bea5-42e9-9a3d-57f535d40ca5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:19.869083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:19.870263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:19.870546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:19.871140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:19.873510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:19.980251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:19.981776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a709393b-3df6-46cd-9073-4c84473710ac service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:19.991627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe28fba8-18a9-4f42-8246-640a70929695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:19.992502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fba8-18a9-4f42-8246-640a70929695 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:19.992853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fba8-18a9-4f42-8246-640a70929695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:19.993500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe28fba8-18a9-4f42-8246-640a70929695 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:19.994243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28fba8-18a9-4f42-8246-640a70929695 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:19.997814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe28fba8-18a9-4f42-8246-640a70929695 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:20.013634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 719/1427] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:20.016330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.017651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.017930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.073996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 709/1428] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:19 2022] POST /identity/v3/projects => generated 377 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:20.074822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.075113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.077173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.078498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:20.079976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.138272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:20.146000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.246142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 720/1429] 10.222.0.43 () {58 vars in 1015 bytes} [Wed Oct 12 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:20.247414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:20.274831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:20.350016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 710/1430] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:20.364791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:20.367152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.464312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f359fdd-7de4-4cfb-a6fd-ceb9ffb55235 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:20.481994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:20.492848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.494029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.495599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.552837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:20.557626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:20.620465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 711/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:20.623375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:20.634036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:20.643633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ddef0-34ba-4c40-b167-ec3ea054c35d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:20.664256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:20.665006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.666987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.668476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.671371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:20.679704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:20.680229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5609bc54-c627-4daf-96f4-fb15b009c42a service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:20.702419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:20.706502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.706821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.707311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.729200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:20.736658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:20.751360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 721/1432] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:20 2022] POST /identity/v3/users => generated 295 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:20.755082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:20.772879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 712/1433] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:20.787604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6a1d26-fb6c-4037-957c-ca92abb7c54e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:20.794217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:20.795679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.796055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.796425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.797235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:20.799180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:20.818790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:20.884729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:20.901300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 722/1434] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:20.920296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe59f2-b5e9-4d01-8e0a-d808f219a965 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:20.928356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:20.930016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:20.930294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:20.930564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] PATH_INFO: `/v3/projects/aa8fc3a3d1a44281a4ff7acdba1e6787/users/6a36e646bce441338698dd64c1050d55/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:20.969134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8fc3a3d1a44281a4ff7acdba1e6787, user_id=6a36e646bce441338698dd64c1050d55, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:20.977825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:21.014465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:21.016355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 713/1435] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:20 2022] PUT /identity/v3/projects/aa8fc3a3d1a44281a4ff7acdba1e6787/users/6a36e646bce441338698dd64c1050d55/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:21.028862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1c36-03da-43b6-9274-18ce618f8a3b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:21.195952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:21.199363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:21.242183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:21.242895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:21.243103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:21.243642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:21.244821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:21.246743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:21.286901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 723/1436] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:21.292198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:21.336051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a364d3-5cd5-4482-b7df-3f452d653c0f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:21.357425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a3469-0c20-4bc2-8205-669b5d1b528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:21.358087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a3469-0c20-4bc2-8205-669b5d1b528a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:21.358323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a3469-0c20-4bc2-8205-669b5d1b528a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:21.358603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a3469-0c20-4bc2-8205-669b5d1b528a admin admin] PATH_INFO: `/v3/projects/aa8fc3a3d1a44281a4ff7acdba1e6787/users/6a36e646bce441338698dd64c1050d55/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:21.360697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a3469-0c20-4bc2-8205-669b5d1b528a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8fc3a3d1a44281a4ff7acdba1e6787, user_id=6a36e646bce441338698dd64c1050d55, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:21.363498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a3469-0c20-4bc2-8205-669b5d1b528a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:21.395738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d638f04d-d1a2-4431-bc07-1c808ba1cac6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:21.418891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 724/1437] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:21 2022] PUT /identity/v3/projects/aa8fc3a3d1a44281a4ff7acdba1e6787/users/6a36e646bce441338698dd64c1050d55/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:21.429401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:21.430070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:21.431371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:21.535007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] MFA Rules not processed for user `6a36e646bce441338698dd64c1050d55`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:21.536421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:21.565291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:21.566258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:21.566538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:21.566834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] PATH_INFO: `/v3/users/2fbf56b734a443ccb2dee9ab463e1d00` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:21.568783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fbf56b734a443ccb2dee9ab463e1d00)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:21.571499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:21.669213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:21.694800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fbf56b734a443ccb2dee9ab463e1d00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:21.699340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fbf56b734a443ccb2dee9ab463e1d00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:21.701058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fbf56b734a443ccb2dee9ab463e1d00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:21.703803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 725/1438] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:21.715586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:21.717169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:21.732699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fbf56b734a443ccb2dee9ab463e1d00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:21.752929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-810679ec-bc28-40a5-80a7-ab7e75e3c76a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fbf56b734a443ccb2dee9ab463e1d00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:21.865327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 714/1439] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:21 2022] DELETE /identity/v3/users/2fbf56b734a443ccb2dee9ab463e1d00 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:21.868081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088926e8-6266-4e75-9b7c-d1db304f689a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:21.874987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:21.875872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:21.876373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:21.918349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:21.926076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:21.933763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:22.018062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:22.019085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:22.019581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:22.020064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:22.028999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:22.133238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 715/1440] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:22.182869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:22.184613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:22.201630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 726/1441] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:22.521093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:22.524298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:22.704219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5457ba7-8777-4516-952e-f5d53e94ca7f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:22.716246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59b36b-301f-49d5-a4bc-7a852a0a08b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:22.721061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b36b-301f-49d5-a4bc-7a852a0a08b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:22.723941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b36b-301f-49d5-a4bc-7a852a0a08b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:22.732120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59b36b-301f-49d5-a4bc-7a852a0a08b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:22.800505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59b36b-301f-49d5-a4bc-7a852a0a08b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:22.805846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59b36b-301f-49d5-a4bc-7a852a0a08b3 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:22.835017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 716/1442] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:23.102388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:23.105423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:23.131107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b86ff3c-6994-4dc3-a33d-7e1b10dfeefd service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:23.144749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:23.145434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.145738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.145987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] PATH_INFO: `/v3/projects/419203e8fefa4cba9f2579f5fa821f46` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.148423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] RBAC: Authorizing `identity:delete_project(project_id=419203e8fefa4cba9f2579f5fa821f46)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:23.153293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:23.328489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '419203e8fefa4cba9f2579f5fa821f46', 'request_id': 'req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:23.346568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '419203e8fefa4cba9f2579f5fa821f46', 'request_id': 'req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:23.368670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '419203e8fefa4cba9f2579f5fa821f46', 'request_id': 'req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:23.436701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] The token cache is being invalidate because project 419203e8fefa4cba9f2579f5fa821f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:23.438629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:23.442679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-10cb0ef3-cec3-4d98-8cfd-a6b56666c62f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:23.450420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 727/1443] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:23 2022] DELETE /identity/v3/projects/419203e8fefa4cba9f2579f5fa821f46 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:23.456397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040198a-72c5-4195-aa82-aad4e1687a4d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.457022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040198a-72c5-4195-aa82-aad4e1687a4d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.457602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040198a-72c5-4195-aa82-aad4e1687a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.485059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4040198a-72c5-4195-aa82-aad4e1687a4d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:23.486910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4040198a-72c5-4195-aa82-aad4e1687a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:23.605122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.605541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.605830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.606646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 717/1444] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:23.655925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:23.657299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:23.673252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.673882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.674331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.688708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:23.692661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 728/1445] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:23.698909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:23.709757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f083985-c866-4b32-8557-73f1d420e025 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:23.716860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:23.717833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.718058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.718394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.719248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:23.719869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:23.721785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:23.739988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] MFA Rules not processed for user `abbc8a34ac8f44328596102c35e184dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:23.744076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 729/1446] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:23.754170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:23.766409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:23.795069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b90076a-6414-494c-9612-43fee5dcdaf9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:23.827321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:23.833171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:23.834786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:23.836026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:23.836853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:23.845900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:23.956682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 718/1447] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:24.007946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 730/1448] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:23 2022] POST /identity/v3/projects => generated 379 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:24.009249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:24.033631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:24.035567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:24.039555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e6284-e3ef-4646-9f26-e31c6d205fa6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:24.082138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:24.196122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76379049-215d-4d55-816e-c7f37e92fabe admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:24.206249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:24.206956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:24.207237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:24.209665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] PATH_INFO: `/v3/projects/da2684ea501046d19767ebf4d17630f6` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:24.210986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da2684ea501046d19767ebf4d17630f6)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:24.218040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:24.304475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:24.305339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:24.305785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:24.306261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:24.307394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:24.309781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:24.357672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 719/1449] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:24 2022] POST /identity/v3/users => generated 290 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:24.371093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:24.389968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e263bb0-c8b7-41c6-b839-eb98f7c71579 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:24.464212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da2684ea501046d19767ebf4d17630f6', 'request_id': 'req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:24.499703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da2684ea501046d19767ebf4d17630f6', 'request_id': 'req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:24.528978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da2684ea501046d19767ebf4d17630f6', 'request_id': 'req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:24.620129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] The token cache is being invalidate because project da2684ea501046d19767ebf4d17630f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:24.624164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:24.625547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:24.629983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 731/1450] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:24 2022] DELETE /identity/v3/projects/da2684ea501046d19767ebf4d17630f6 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:31:24.634688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:24.639533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:24.818459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:24.819383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:24.820177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:24.821633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:24.823146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:24.829567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:24.873638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 720/1451] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 507 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:24.875493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:24.881719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:24.940805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c3dc9-029c-4083-8add-ec18b23b03d7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:24.999378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:25.006571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:25.006889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:25.007171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:25.017022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:25.023019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:25.066002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 721/1452] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:24 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:25.067033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:25.067412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4f23b-faaa-4aa0-a8f0-42c3e2cee2e2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:25.097644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90578ae3-63c7-4e77-a2e6-67d73840194a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:25.098323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:25.101390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:25.101678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:25.101959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] PATH_INFO: `/v3/users/a80fd9a75cc448d48e7adaeebafb31f9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:25.107121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a80fd9a75cc448d48e7adaeebafb31f9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:25.113740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:25.114052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a286f4-bd2f-4283-8e27-9f3e809be479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:25.114744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a286f4-bd2f-4283-8e27-9f3e809be479 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:25.115028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a286f4-bd2f-4283-8e27-9f3e809be479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:25.115304 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a286f4-bd2f-4283-8e27-9f3e809be479 admin admin] PATH_INFO: `/v3/projects/2696212804034d569b4f24b2f6e3ae55/users/e6f8149f7a484b4190b8be9394222eea/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:25.201101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a286f4-bd2f-4283-8e27-9f3e809be479 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2696212804034d569b4f24b2f6e3ae55, user_id=e6f8149f7a484b4190b8be9394222eea, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:25.204262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a286f4-bd2f-4283-8e27-9f3e809be479 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:25.240852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:25.266021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 722/1453] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:25 2022] PUT /identity/v3/projects/2696212804034d569b4f24b2f6e3ae55/users/e6f8149f7a484b4190b8be9394222eea/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:25.268781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:25.268781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:25.268781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:25.309758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] MFA Rules not processed for user `e6f8149f7a484b4190b8be9394222eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:25.311008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:25.354967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a80fd9a75cc448d48e7adaeebafb31f9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:25.362105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:25.377999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a80fd9a75cc448d48e7adaeebafb31f9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:25.378974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a80fd9a75cc448d48e7adaeebafb31f9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:25.406375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a80fd9a75cc448d48e7adaeebafb31f9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:25.408644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a80fd9a75cc448d48e7adaeebafb31f9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:25.464746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 732/1454] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:24 2022] DELETE /identity/v3/users/a80fd9a75cc448d48e7adaeebafb31f9 => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:25.501008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 723/1455] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:25.505812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:25.507509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:25.643213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f45b486-4661-416f-aa0f-6b527c4850a7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:25.656402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd653242-bf0b-4efb-b6de-2079fbed1f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:25.657330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd653242-bf0b-4efb-b6de-2079fbed1f7f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:25.657750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd653242-bf0b-4efb-b6de-2079fbed1f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:25.658143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd653242-bf0b-4efb-b6de-2079fbed1f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:25.659051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd653242-bf0b-4efb-b6de-2079fbed1f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:25.661262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd653242-bf0b-4efb-b6de-2079fbed1f7f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:25.718565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 733/1456] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:25 2022] POST /identity/v3/projects => generated 379 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:25.724559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:26.138799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8db13-0ef0-4886-9f5c-70516a289c92 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.139185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8db13-0ef0-4886-9f5c-70516a289c92 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.139619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8db13-0ef0-4886-9f5c-70516a289c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.217904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-39c8db13-0ef0-4886-9f5c-70516a289c92 None None] MFA Rules not processed for user `383004527ce84e4f9fc40760a040936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:26.222206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39c8db13-0ef0-4886-9f5c-70516a289c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:26.374820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4307a68a-db42-4b6f-a022-9f78baee758d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:26.446939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 734/1457] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:26.449591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c3d94-bc37-48f7-b010-13698be7bdaa None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.450106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c3d94-bc37-48f7-b010-13698be7bdaa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.450529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c3d94-bc37-48f7-b010-13698be7bdaa None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.452835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 735/1458] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:31:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:26.491381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.491884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.492456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.551068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:26.552116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.552242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:26.552773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.554809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.555130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:26.556107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:26.558581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:26.599996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 724/1459] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:25 2022] POST /identity/v3/users => generated 290 bytes in 877 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:26.601206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:26.610282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:26.642368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a8a90-fcc0-4526-a2ea-b7195f00aef2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:26.663651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:26.664560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.665067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.665591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] PATH_INFO: `/v3/projects/4bfc5899b0874582b1d4721c7741bc8d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.669045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bfc5899b0874582b1d4721c7741bc8d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:26.671190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:26.673513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:26.714500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 736/1460] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:31:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:26.715626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:26.727839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68df2e07-0186-413c-9c11-9b965ea0ff39 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:26.735414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:26.736229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.737552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:26.738373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.738924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.739812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:26.742439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:26.782493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 737/1461] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:26.785507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:26.800396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bfc5899b0874582b1d4721c7741bc8d', 'request_id': 'req-e2f21b48-6e7e-4430-8e1e-106b5dac795c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:26.814366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9a3c4-a944-4e89-a155-82f4f9a81722 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:26.821856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bfc5899b0874582b1d4721c7741bc8d', 'request_id': 'req-e2f21b48-6e7e-4430-8e1e-106b5dac795c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:26.844840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bfc5899b0874582b1d4721c7741bc8d', 'request_id': 'req-e2f21b48-6e7e-4430-8e1e-106b5dac795c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:26.890688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] The token cache is being invalidate because project 4bfc5899b0874582b1d4721c7741bc8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:26.892167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:26.894314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:26.899615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 725/1462] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:26 2022] DELETE /identity/v3/projects/4bfc5899b0874582b1d4721c7741bc8d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:26.907289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:26.910243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:26.959570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:26.960206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:26.961329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:26.961627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:26.962292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:26.964580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:26.994384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 738/1463] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:26.999111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:27.001838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.057734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a02572-b9d4-455b-842e-18277f3b6061 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.071064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd6232-cea8-4ab7-92e4-4068340b9fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.071752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd6232-cea8-4ab7-92e4-4068340b9fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.081683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd6232-cea8-4ab7-92e4-4068340b9fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.082004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd6232-cea8-4ab7-92e4-4068340b9fb4 admin admin] PATH_INFO: `/v3/projects/25238362927a4e7c9c16d32378ab6cf4/users/10ba3890cc104ce092639f19e6057bbd/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.102591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f21b48-6e7e-4430-8e1e-106b5dac795c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.131166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.132283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.132805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.133246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] PATH_INFO: `/v3/users/36c436a9a6d945158df1efaea3058bf1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.134910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36c436a9a6d945158df1efaea3058bf1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.137917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:27.142579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd6232-cea8-4ab7-92e4-4068340b9fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25238362927a4e7c9c16d32378ab6cf4, user_id=10ba3890cc104ce092639f19e6057bbd, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.145286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd6232-cea8-4ab7-92e4-4068340b9fb4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:27.180756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 739/1464] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:26 2022] PUT /identity/v3/projects/25238362927a4e7c9c16d32378ab6cf4/users/10ba3890cc104ce092639f19e6057bbd/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:27.183459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.183793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.184127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.246772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36c436a9a6d945158df1efaea3058bf1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.247041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] MFA Rules not processed for user `10ba3890cc104ce092639f19e6057bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:27.248277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.248770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36c436a9a6d945158df1efaea3058bf1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.249579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36c436a9a6d945158df1efaea3058bf1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.263290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36c436a9a6d945158df1efaea3058bf1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.280285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36c436a9a6d945158df1efaea3058bf1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.310767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 726/1465] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:26 2022] DELETE /identity/v3/users/36c436a9a6d945158df1efaea3058bf1 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:27.310767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:27.311144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.365871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:27.419370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 740/1466] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:27.420257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:27.421835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.444976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa9feb-13ec-431b-82e6-ec05a313e6b5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.451178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.451921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.452278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.452660 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] PATH_INFO: `/v3/projects/2947df8b24c04bc5b86a08b6f56f97b0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.454054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2947df8b24c04bc5b86a08b6f56f97b0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.457281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:27.545740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:27.561466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2947df8b24c04bc5b86a08b6f56f97b0', 'request_id': 'req-72d73137-f918-4946-a587-4b94a4750962'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.573544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2947df8b24c04bc5b86a08b6f56f97b0', 'request_id': 'req-72d73137-f918-4946-a587-4b94a4750962'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.583163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2947df8b24c04bc5b86a08b6f56f97b0', 'request_id': 'req-72d73137-f918-4946-a587-4b94a4750962'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.595833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f41103-e32d-452f-9cb5-ef33b51caa29 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.619000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] The token cache is being invalidate because project 2947df8b24c04bc5b86a08b6f56f97b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:27.620199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.621161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-72d73137-f918-4946-a587-4b94a4750962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:27.623303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 727/1467] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:27 2022] DELETE /identity/v3/projects/2947df8b24c04bc5b86a08b6f56f97b0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:27.628909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e618fef-e53b-4f74-9b30-95542ea30348 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.629288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e618fef-e53b-4f74-9b30-95542ea30348 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.629669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e618fef-e53b-4f74-9b30-95542ea30348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.653528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9e618fef-e53b-4f74-9b30-95542ea30348 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:27.654714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9e618fef-e53b-4f74-9b30-95542ea30348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.686689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.687725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.688146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.688497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.689929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.692329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:27.737079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 741/1468] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:27 2022] POST /identity/v3/projects => generated 379 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:27.739112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:27.741639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.752222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 728/1469] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:27.753776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.754047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.754333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.780638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:27.782261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:27.905595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:27.933686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8967a0de-21e3-4214-aae4-2593584de6d1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.934857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 729/1470] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:27.939504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:27.943082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.943953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.944334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.944713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.954055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c741957-3eb9-428e-aeec-718b27566a18 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:27.954369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.956547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:27.973887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:27.975150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:27.975533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:27.975912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:27.976689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:27.977102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:27.979212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.018470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 742/1471] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:27 2022] POST /identity/v3/users => generated 296 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:28.024846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 730/1472] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:28.026256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.027366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.049853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cf0b9b-88e4-4365-bb67-33f90fedf48b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.051956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32dd9d3-6263-4592-a8ff-b0c29a21aca6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.064491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.065493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.066648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.067389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.067782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.068218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.068676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.069103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.069946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.071136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.076051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.077239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.105178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 743/1473] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:28.107227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.110959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.160453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 731/1474] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:28 2022] POST /identity/v3/projects => generated 363 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:28.163501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.168325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.282440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54c21c-e63d-4efa-820b-a3dad868667f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.283485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:28.304070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.304930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.305645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.306001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] PATH_INFO: `/v3/projects/c1065b83822f4207b293673e8ba99941` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.310995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1065b83822f4207b293673e8ba99941)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.313082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.340375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827adec-412d-46fa-8b54-dfae99dde4df admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.385661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1065b83822f4207b293673e8ba99941', 'request_id': 'req-700a84b1-4631-4c51-a870-206691dd3b80'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:28.412588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1065b83822f4207b293673e8ba99941', 'request_id': 'req-700a84b1-4631-4c51-a870-206691dd3b80'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:28.424715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1065b83822f4207b293673e8ba99941', 'request_id': 'req-700a84b1-4631-4c51-a870-206691dd3b80'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:28.446744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] The token cache is being invalidate because project c1065b83822f4207b293673e8ba99941 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:28.447349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:28.447702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:28.450451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 732/1475] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:28 2022] DELETE /identity/v3/projects/c1065b83822f4207b293673e8ba99941 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:28.453914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.456369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.480595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.481392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.481675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.482017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.483869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.499027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.501499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e79269-2b7a-47f1-8467-dce66ed223d0 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.515860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 744/1476] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:28.519881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1448b-b8e3-4745-9a15-6aca489091e9 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.520302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1448b-b8e3-4745-9a15-6aca489091e9 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.520615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a1448b-b8e3-4745-9a15-6aca489091e9 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.522436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 745/1477] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:28.523163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1448b-b8e3-4745-9a15-6aca489091e9 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.525124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-75a1448b-b8e3-4745-9a15-6aca489091e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.564468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:28.598983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700a84b1-4631-4c51-a870-206691dd3b80 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.612650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43297265-363b-46ba-a28f-1d632bf79f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.613173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a1448b-b8e3-4745-9a15-6aca489091e9 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.616341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43297265-363b-46ba-a28f-1d632bf79f8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.616841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43297265-363b-46ba-a28f-1d632bf79f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.617372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43297265-363b-46ba-a28f-1d632bf79f8b admin admin] PATH_INFO: `/v3/projects/e3cc0826c6b441539b7db4d71d151b1a/users/20fd4ae35e5243fb8d0e61c87bb78d45/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.621138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a0e83-7557-4676-9216-93b740ce0de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.622085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a0e83-7557-4676-9216-93b740ce0de7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.622439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a0e83-7557-4676-9216-93b740ce0de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.622842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a0e83-7557-4676-9216-93b740ce0de7 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.623980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a0e83-7557-4676-9216-93b740ce0de7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.626420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a0e83-7557-4676-9216-93b740ce0de7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.660746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43297265-363b-46ba-a28f-1d632bf79f8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cc0826c6b441539b7db4d71d151b1a, user_id=20fd4ae35e5243fb8d0e61c87bb78d45, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.662318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 746/1478] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:28 2022] POST /identity/v3/users => generated 282 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:28.663773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.664161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.664388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.674551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43297265-363b-46ba-a28f-1d632bf79f8b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.699654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:28.700932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.727578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 733/1479] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:28 2022] PUT /identity/v3/projects/e3cc0826c6b441539b7db4d71d151b1a/users/20fd4ae35e5243fb8d0e61c87bb78d45/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:28.733184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.734641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.734994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.773915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 747/1480] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:28.775276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.781475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:28.782731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.793554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:28.804899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1810718a-cc5c-44a6-b444-79f5a4106a6a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.819539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.820166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.820450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.820729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.821407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.825065 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.839693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 748/1481] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:28.839693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.860979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ad4bda-28e7-4c69-bc57-fd734639b31e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.866954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e946a9-fced-402d-b5e1-da18a0115007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.867569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e946a9-fced-402d-b5e1-da18a0115007 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.868511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e946a9-fced-402d-b5e1-da18a0115007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.868968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e946a9-fced-402d-b5e1-da18a0115007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.870536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e946a9-fced-402d-b5e1-da18a0115007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.874026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e946a9-fced-402d-b5e1-da18a0115007 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.900276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 749/1482] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:28.900276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.900276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.900276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.920192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 734/1483] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:28.921198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.928503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:28.929508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.933615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bc936-585e-49a5-b0f6-1773bf506b2b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.939181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.939845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.940127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.940399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.941098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.943010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.943429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 750/1484] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:28.944336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.959658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 735/1485] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:28.960516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:28.962294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:28.965301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce375b69-565d-437e-9091-d20ae053a2dc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.970221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.971260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.971698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.972043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] PATH_INFO: `/v3/projects/e3cc0826c6b441539b7db4d71d151b1a/users/20fd4ae35e5243fb8d0e61c87bb78d45/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.974918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cc0826c6b441539b7db4d71d151b1a, user_id=20fd4ae35e5243fb8d0e61c87bb78d45, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:28.977005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:28.979623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcce983-c956-47cb-a2ff-d7247251e588 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:28.986485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:28.987097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:28.987375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:28.987644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:28.989926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 751/1486] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:28 2022] PUT /identity/v3/projects/e3cc0826c6b441539b7db4d71d151b1a/users/20fd4ae35e5243fb8d0e61c87bb78d45/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:28.990866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.007482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.009239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.020574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 736/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:29.021213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.032071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d82d-858f-4eb1-81d6-b974ab6dc9f9 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.078714 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c48aa-310d-48f9-b5d8-94f8e77c1167 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.084904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.086161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.086442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.086729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.087441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:29.087766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.090727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5d2f1-54af-4128-878d-08ea54cd2484 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.103388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 752/1488] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:29.106706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f61366-b0df-4fa5-ad1e-90469dc52634 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.107512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f61366-b0df-4fa5-ad1e-90469dc52634 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.107942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f61366-b0df-4fa5-ad1e-90469dc52634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.110912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.111573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.111855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.112130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] PATH_INFO: `/v3/projects/e62f3c60fa5d4bd283ada2dacb3d73a5/users/18cf47c3c9314623ad5956e95454eee6/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.128192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e62f3c60fa5d4bd283ada2dacb3d73a5, user_id=18cf47c3c9314623ad5956e95454eee6, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.130933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.136920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-41f61366-b0df-4fa5-ad1e-90469dc52634 None None] MFA Rules not processed for user `20fd4ae35e5243fb8d0e61c87bb78d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:29.138247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-41f61366-b0df-4fa5-ad1e-90469dc52634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:29.144530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 737/1489] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:29 2022] PUT /identity/v3/projects/e62f3c60fa5d4bd283ada2dacb3d73a5/users/18cf47c3c9314623ad5956e95454eee6/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:29.145458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.183511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:29.211201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 753/1490] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:29.212379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.212666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.213131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.221276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab8a37-f451-41e3-badc-c76b87ed0adb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.226332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.227040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.228046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.228325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.229513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.231565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.242046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] MFA Rules not processed for user `18cf47c3c9314623ad5956e95454eee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:29.243525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:29.262631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 738/1491] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/projects => generated 357 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:29.263919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.267195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:29.283018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:29.319446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 754/1492] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:29.320261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.332205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d58ec7b-296b-4525-aeb3-0a1804f00a54 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.332656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a62a4b-8f53-46de-a1f2-767ac81df777 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.336526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.337293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.337707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.338744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.396453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.397070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.397370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.397644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.398276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.398975 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.400238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.401101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.410600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 739/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:29.411462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.422824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f5c46-d01d-4503-8f38-34e95cfba126 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.425754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 755/1494] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/users => generated 279 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:29.429456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.430527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.431128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.431400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.431734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.432368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.434513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.441289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff75d4-ba9a-4484-b329-03e44edde201 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.446950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.447642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.447920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.448192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.448835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.450463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.465255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 756/1495] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:31:29.465255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.467627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:29.476316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 740/1496] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/projects => generated 363 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:29.478705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.482336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a478bc-e668-4e51-a848-bd38361e6b2b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.524712 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:29.540515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.542153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.542436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.543467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.544525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:29.547175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb5c4a8-9313-46cd-910f-42d35947f93b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.552886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.553616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.553969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.554309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.555206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.557689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.573476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 741/1497] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:29.574989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.586439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc5a8-2b45-4a28-97f2-476ced0ad4ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.592197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.592789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.593070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.593486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.594118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.596049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.612752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.614640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.627677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 742/1498] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:29.627677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.629141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 757/1499] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:29.630245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.636837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73e66a8-a1d4-4db9-9fe0-cc36c80bc81b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.641803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4f0dc-90a8-44e3-b734-13da65959cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.642547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f0dc-90a8-44e3-b734-13da65959cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.642889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f0dc-90a8-44e3-b734-13da65959cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.643233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4f0dc-90a8-44e3-b734-13da65959cb6 admin admin] PATH_INFO: `/v3/projects/a0e18d53c13f46bf8dbb5ff9588f6366/users/b6852a16561048beba96e8cc899df6a7/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.646980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:29.655033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0801459d-a336-461e-b7cb-48b103776084 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.660682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.661073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.661400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.661674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.662315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.663903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.664404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f0dc-90a8-44e3-b734-13da65959cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0e18d53c13f46bf8dbb5ff9588f6366, user_id=b6852a16561048beba96e8cc899df6a7, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.669739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4f0dc-90a8-44e3-b734-13da65959cb6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.684835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 758/1500] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:29.686015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.691004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 743/1501] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:29 2022] PUT /identity/v3/projects/a0e18d53c13f46bf8dbb5ff9588f6366/users/b6852a16561048beba96e8cc899df6a7/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:29.701752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.702105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.702385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.706467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f6916-4e75-494d-a24a-69bb5c958f46 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:29.730962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] MFA Rules not processed for user `b6852a16561048beba96e8cc899df6a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:29.732187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:29.779194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:29.780040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:29.780329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:29.780753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] PATH_INFO: `/v3/projects/d63c543aadfd421cbf280bda36d7e2e0/users/385e43e4979543828b940e6d71362992/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:29.803517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63c543aadfd421cbf280bda36d7e2e0, user_id=385e43e4979543828b940e6d71362992, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:29.811435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:29.859831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 744/1502] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:29.861341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.862397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 759/1503] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:29 2022] PUT /identity/v3/projects/d63c543aadfd421cbf280bda36d7e2e0/users/385e43e4979543828b940e6d71362992/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:31:29.866947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:29.892182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf61e0f-e164-4c9b-a906-60ad7ffae556 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.021479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba168422-09c3-4444-ab48-b183c8adec98 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.034266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.036867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.037412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.038708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.040860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.047088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.055615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.056250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.059477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.059592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.060164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.065947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.073662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 760/1504] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:29 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:30.088716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.112202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 745/1505] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:29 2022] POST /identity/v3/projects => generated 357 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:30.114357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4609149-6a25-4f96-b4a3-23bd6e4e8922 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.116279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.130752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:30.223984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991b085-665c-4958-98a1-35ac1933da09 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.239066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.239754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.240137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.240362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.241001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.245994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.256534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a2d61-2c95-4bdd-af3c-1d1d2da02ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.263900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a2d61-2c95-4bdd-af3c-1d1d2da02ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.264656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a2d61-2c95-4bdd-af3c-1d1d2da02ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.264656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a2d61-2c95-4bdd-af3c-1d1d2da02ffc admin admin] PATH_INFO: `/v3/projects/d63c543aadfd421cbf280bda36d7e2e0/users/385e43e4979543828b940e6d71362992/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.267041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a2d61-2c95-4bdd-af3c-1d1d2da02ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d63c543aadfd421cbf280bda36d7e2e0, user_id=385e43e4979543828b940e6d71362992, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.269769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a2d61-2c95-4bdd-af3c-1d1d2da02ffc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.282012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 746/1506] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:30 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:30.286199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.287347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 761/1507] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:30 2022] PUT /identity/v3/projects/d63c543aadfd421cbf280bda36d7e2e0/users/385e43e4979543828b940e6d71362992/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:30.298973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.299265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.299587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.314430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed20c08-2b84-4547-9907-858a8838e515 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.323000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:30.350228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] MFA Rules not processed for user `385e43e4979543828b940e6d71362992`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:30.352101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:30.428032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.429149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.429794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.430176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.430968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.433167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.445517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 747/1508] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:30.446471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.459490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 762/1509] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:30.459490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.471776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39905a79-eed7-4df7-9b85-38f648fba6e7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.476286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.476965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.477240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.477538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] PATH_INFO: `/v3/projects/2673617b81a24d80b7335d1238a38c42/users/b35522c5faff4b80a80421fd2a38486f/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.493973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2673617b81a24d80b7335d1238a38c42, user_id=b35522c5faff4b80a80421fd2a38486f, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.497192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.505486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5aca9-efc2-40e0-82e2-5f150f2a4059 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.512957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 763/1510] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:30 2022] PUT /identity/v3/projects/2673617b81a24d80b7335d1238a38c42/users/b35522c5faff4b80a80421fd2a38486f/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:30.514035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.573785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.574521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.574869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.575212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.577037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:30.579314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2b9ae-04d0-48bb-8133-a399a7b14191 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.582402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:30.583488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.583993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.584417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.584756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.652323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.653820 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.664109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 748/1511] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:30.664945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.675216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.677661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.691569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 764/1512] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:30.787582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f58000-6a51-437c-bcc1-245ad83215e7 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.797243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.805464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.805464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.805464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.805464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.805934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.805934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.817249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc042d8-1cc4-44e5-9c74-62b310ac69b7 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.827909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.828592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.828885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.829162 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.844268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 749/1513] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:30.849156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:30.853471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:30.879427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7bf01-6fc4-4374-a87d-50fe7378b8ab admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:30.893552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a920a2-b7f8-4835-888e-f708631422a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:30.894243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a920a2-b7f8-4835-888e-f708631422a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.894525 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a920a2-b7f8-4835-888e-f708631422a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.895399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a920a2-b7f8-4835-888e-f708631422a1 admin admin] PATH_INFO: `/v3/projects/2673617b81a24d80b7335d1238a38c42/users/b35522c5faff4b80a80421fd2a38486f/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.897348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a920a2-b7f8-4835-888e-f708631422a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2673617b81a24d80b7335d1238a38c42, user_id=b35522c5faff4b80a80421fd2a38486f, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.910020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a920a2-b7f8-4835-888e-f708631422a1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.951480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:30.953143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:30.959510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 750/1514] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:30 2022] PUT /identity/v3/projects/2673617b81a24d80b7335d1238a38c42/users/b35522c5faff4b80a80421fd2a38486f/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:30.960742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:30.960940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:30.961211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:30.970038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 765/1515] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:31.043464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] MFA Rules not processed for user `b35522c5faff4b80a80421fd2a38486f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:31.045181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:31.333677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 751/1516] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:31.387058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:31.401167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f100de-8435-4d53-92c3-01ab348d82f6 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:31.488877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:31.490447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:31.490972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:31.491516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:31.519574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:31.521143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:31.541513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 766/1517] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:31 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:31.662019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:31.677655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cc1f9-a3d4-4570-845a-7fa905267411 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:31.681391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c8070-3a5e-4811-9094-14dc52be86c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:31.682517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c8070-3a5e-4811-9094-14dc52be86c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:31.682791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c8070-3a5e-4811-9094-14dc52be86c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:31.683051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c8070-3a5e-4811-9094-14dc52be86c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:31.688970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:31.788934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c8070-3a5e-4811-9094-14dc52be86c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:31.790691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c8070-3a5e-4811-9094-14dc52be86c5 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:31.805297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 752/1518] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:39.586090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:39.652451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b9b6d-1887-4b36-84a1-5ca2b0d7616b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:39.657847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:39.658549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:39.658850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:39.659166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] PATH_INFO: `/v3/users/e6f8149f7a484b4190b8be9394222eea` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:39.660437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6f8149f7a484b4190b8be9394222eea)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:39.662618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:39.712309 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6f8149f7a484b4190b8be9394222eea'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:39.719972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6f8149f7a484b4190b8be9394222eea'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:39.720777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6f8149f7a484b4190b8be9394222eea'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:39.728954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:39.731320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6f8149f7a484b4190b8be9394222eea'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:39.732224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6f8149f7a484b4190b8be9394222eea'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:39.754847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 767/1519] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:39 2022] DELETE /identity/v3/users/e6f8149f7a484b4190b8be9394222eea => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:39.759313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:39.759964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:39.759964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:39.780634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:39.781886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:39.849027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 753/1520] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:39.855812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:39.857289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:39.929976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb614a-0988-49cf-b70e-e9220991e816 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:39.934446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:39.935061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:39.935340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:39.935611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:39.944541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:39.946001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:39.955380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 768/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:31:40.235697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:40.237766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:40.251918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54eeb-f53f-42be-a35f-adafc48dfc1c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:40.257961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:40.258605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:40.258887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:40.259183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] PATH_INFO: `/v3/projects/2696212804034d569b4f24b2f6e3ae55` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:40.260846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2696212804034d569b4f24b2f6e3ae55)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:40.274144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:40.342035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2696212804034d569b4f24b2f6e3ae55', 'request_id': 'req-75639809-a4d0-446a-87e8-125bdf588751'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.354141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2696212804034d569b4f24b2f6e3ae55', 'request_id': 'req-75639809-a4d0-446a-87e8-125bdf588751'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.363968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2696212804034d569b4f24b2f6e3ae55', 'request_id': 'req-75639809-a4d0-446a-87e8-125bdf588751'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.376901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] The token cache is being invalidate because project 2696212804034d569b4f24b2f6e3ae55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:40.377202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.378404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.380036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 754/1522] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:40 2022] DELETE /identity/v3/projects/2696212804034d569b4f24b2f6e3ae55 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:40.384470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:40.396330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a55fdc9-f933-46a2-99ce-6b2f3a1e0c54 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:40.462358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:40.463458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:40.463748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:40.464107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] PATH_INFO: `/v3/users/10ba3890cc104ce092639f19e6057bbd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:40.465267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] RBAC: Authorizing `identity:delete_user(user_id=10ba3890cc104ce092639f19e6057bbd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:40.467222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:40.521416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10ba3890cc104ce092639f19e6057bbd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.530247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10ba3890cc104ce092639f19e6057bbd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.530882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10ba3890cc104ce092639f19e6057bbd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.542645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10ba3890cc104ce092639f19e6057bbd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.543375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10ba3890cc104ce092639f19e6057bbd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:40.558882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:40.563883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 769/1523] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:40 2022] DELETE /identity/v3/users/10ba3890cc104ce092639f19e6057bbd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:40.908761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:40.910281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:40.923837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75639809-a4d0-446a-87e8-125bdf588751 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:40.958512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:40.997989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:40.998851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:40.999133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:40.999398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] PATH_INFO: `/v3/projects/25238362927a4e7c9c16d32378ab6cf4` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.000514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] RBAC: Authorizing `identity:delete_project(project_id=25238362927a4e7c9c16d32378ab6cf4)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:41.002507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:41.051927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25238362927a4e7c9c16d32378ab6cf4', 'request_id': 'req-70218760-5682-4916-8e25-c0db1d17c36a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.060244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25238362927a4e7c9c16d32378ab6cf4', 'request_id': 'req-70218760-5682-4916-8e25-c0db1d17c36a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.066837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25238362927a4e7c9c16d32378ab6cf4', 'request_id': 'req-70218760-5682-4916-8e25-c0db1d17c36a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.077664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] The token cache is being invalidate because project 25238362927a4e7c9c16d32378ab6cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:41.077956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.078621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.080128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 755/1524] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:40 2022] DELETE /identity/v3/projects/25238362927a4e7c9c16d32378ab6cf4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:41.084124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:41.085664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:41.099170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c983e4-3ab3-4bdd-b673-ce08c3d6781d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:41.157337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:41.157883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.158158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.158428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] PATH_INFO: `/v3/users/20fd4ae35e5243fb8d0e61c87bb78d45` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.159523 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20fd4ae35e5243fb8d0e61c87bb78d45)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:41.161488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:41.206393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20fd4ae35e5243fb8d0e61c87bb78d45'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.213537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20fd4ae35e5243fb8d0e61c87bb78d45'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.214189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20fd4ae35e5243fb8d0e61c87bb78d45'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.221210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20fd4ae35e5243fb8d0e61c87bb78d45'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.221867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-25e6afef-a371-4814-a6a7-f6436502d09f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20fd4ae35e5243fb8d0e61c87bb78d45'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.239304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 770/1525] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:31:41 2022] DELETE /identity/v3/users/20fd4ae35e5243fb8d0e61c87bb78d45 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:41.529220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:41.530800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:41.543991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218760-5682-4916-8e25-c0db1d17c36a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:41.602695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:41.603497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.604441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.604441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] PATH_INFO: `/v3/projects/e3cc0826c6b441539b7db4d71d151b1a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.605612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3cc0826c6b441539b7db4d71d151b1a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:41.607587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:41.657614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3cc0826c6b441539b7db4d71d151b1a', 'request_id': 'req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.666950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3cc0826c6b441539b7db4d71d151b1a', 'request_id': 'req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.673790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3cc0826c6b441539b7db4d71d151b1a', 'request_id': 'req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.676613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:41.698574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] The token cache is being invalidate because project e3cc0826c6b441539b7db4d71d151b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:31:41.698950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.699666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6da17d2b-76b5-40a5-a3f6-c9e28ad7d768 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:31:41.701924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 756/1526] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:31:41 2022] DELETE /identity/v3/projects/e3cc0826c6b441539b7db4d71d151b1a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:41.707810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.708099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.708374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.728219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:41.729982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:41.789342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 771/1527] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:41.798634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.798921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.799196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.820270 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:41.821507 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:41.829351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 757/1528] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:41.833468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:41.843529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a0211b-6658-4729-8fae-d0500e007f45 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:41.848207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:41.848814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.849099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.849404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.850054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:31:41.850358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:41.851954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:41.864363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 772/1529] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:31:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:31:41.869054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:41.879206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5c726-0450-4372-af12-27613dd728b5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:41.883833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:41.884425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.884754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.885094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.885816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:41.887696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:41.914272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 758/1530] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:41 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:41.918972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:41.928664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570e769-692a-44fd-9375-f79648e11b2c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:41.932763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:41.988995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:41.989641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:41.989941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:41.990202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:41.990823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:41.992879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:42.016993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 773/1531] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:41 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:42.021157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:42.033924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834d7cd3-61d4-46ae-8dcb-ba3c6e756ff0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:42.039048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:42.039753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:42.040095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:42.040530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:42.042338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:42.043517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:42.054672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 759/1532] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:42.058609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:42.068870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc79358-46e5-4ba1-8b47-7beba52f0b2a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:42.074614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84567c-73bd-4d38-8154-6e9ccdbff093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:42.075220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84567c-73bd-4d38-8154-6e9ccdbff093 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:42.075532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84567c-73bd-4d38-8154-6e9ccdbff093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:42.075751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84567c-73bd-4d38-8154-6e9ccdbff093 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:42.076390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84567c-73bd-4d38-8154-6e9ccdbff093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:42.078002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84567c-73bd-4d38-8154-6e9ccdbff093 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:42.089510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 774/1533] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:42.093711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:42.104055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffdf93-dcce-420d-9cc5-790b563a6f6f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:42.108814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:42.109473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:42.109750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:42.110018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] PATH_INFO: `/v3/projects/df0cedf742a54210a64ad18a30437959/users/aba55e3e1a4048abb617c3063f227dbe/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:42.125517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] RBAC: Authorizing `identity:create_grant(project_id=df0cedf742a54210a64ad18a30437959, user_id=aba55e3e1a4048abb617c3063f227dbe, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:42.128280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:42.143573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 760/1534] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:42 2022] PUT /identity/v3/projects/df0cedf742a54210a64ad18a30437959/users/aba55e3e1a4048abb617c3063f227dbe/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:42.148394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f55807-6687-4b87-a93c-306ea7c8979d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:42.148830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f55807-6687-4b87-a93c-306ea7c8979d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:42.149179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f55807-6687-4b87-a93c-306ea7c8979d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:42.171285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-87f55807-6687-4b87-a93c-306ea7c8979d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:42.173072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-87f55807-6687-4b87-a93c-306ea7c8979d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:42.246147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 775/1535] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:42.255845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:42.261984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:42.326313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061a4963-d62f-4852-ab24-80230613858c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:42.329958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:42.330562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:42.330843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:42.331112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:42.341509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:42.342951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:42.352818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 761/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:43.070765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.071063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.071347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.107690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] MFA Rules not processed for user `aba55e3e1a4048abb617c3063f227dbe`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:43.109094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:43.210284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 776/1537] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:43.215073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:43.225186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877d228-968c-4e1c-a023-46e8df25a2d3 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:43.231051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:43.231653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.232160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.232500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.233264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:43.235398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:43.263747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 762/1538] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:31:43 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:31:43.267479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:43.279525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8054bf-c92e-4c72-b8b9-0fdebfd2b20e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:43.290776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:43.339747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:43.340727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.340727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.340936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.341555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:43.343476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:43.367296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 777/1539] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:31:43 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:31:43.368740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:43.379269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f77de2-647c-49cd-b163-5da16945be09 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:43.384361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:43.385728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.386128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.386556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.387196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:43.388786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:43.400248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 763/1540] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:43.405371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:43.415616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16210884-cd91-4876-83d1-63426812d579 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:43.420432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:43.421088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.421398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.421673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] PATH_INFO: `/v3/projects/83fda30f213a4d80b5ae0ce7bf67417c/users/a48fbc9cad4342f6bcc464758319b71e/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.436943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fda30f213a4d80b5ae0ce7bf67417c, user_id=a48fbc9cad4342f6bcc464758319b71e, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:43.441903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:43.457953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 778/1541] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:43 2022] PUT /identity/v3/projects/83fda30f213a4d80b5ae0ce7bf67417c/users/a48fbc9cad4342f6bcc464758319b71e/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:43.461830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:43.472994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be977be-e55b-4ebb-86ad-166230a6a770 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:43.524290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:31:43.531744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfce27e-5356-491e-8eb6-0ffb428a677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:43.532443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce27e-5356-491e-8eb6-0ffb428a677e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.532959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce27e-5356-491e-8eb6-0ffb428a677e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.533279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfce27e-5356-491e-8eb6-0ffb428a677e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.534378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfce27e-5356-491e-8eb6-0ffb428a677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:43.536338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfce27e-5356-491e-8eb6-0ffb428a677e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:43.546183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 764/1542] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:31:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:31:43.550315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:43.561438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa374c9-d04c-4d1b-8450-193de1c82a0f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:43.565900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:43.566544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:43.566827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:43.567105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] PATH_INFO: `/v3/projects/83fda30f213a4d80b5ae0ce7bf67417c/users/a48fbc9cad4342f6bcc464758319b71e/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:43.569031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83fda30f213a4d80b5ae0ce7bf67417c, user_id=a48fbc9cad4342f6bcc464758319b71e, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:43.571723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:43.582608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 779/1543] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:31:43 2022] PUT /identity/v3/projects/83fda30f213a4d80b5ae0ce7bf67417c/users/a48fbc9cad4342f6bcc464758319b71e/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:31:44.257216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:44.257775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:44.257860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:44.309092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] MFA Rules not processed for user `a48fbc9cad4342f6bcc464758319b71e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:44.312005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:44.421605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 765/1544] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:31:44.449362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:44.451266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:44.509124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94b6d9-afaf-43f2-b848-e6d7798e85f4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:44.513766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663f7eab-ad9b-49b4-8da4-732cbbb11e97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:44.514364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f7eab-ad9b-49b4-8da4-732cbbb11e97 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:44.514641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f7eab-ad9b-49b4-8da4-732cbbb11e97 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:44.514906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f7eab-ad9b-49b4-8da4-732cbbb11e97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:44.570945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f7eab-ad9b-49b4-8da4-732cbbb11e97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:44.572966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663f7eab-ad9b-49b4-8da4-732cbbb11e97 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:44.582019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 780/1545] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:51.217470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:51.227511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3553dbb9-b8e8-4bd1-a5d5-b9562972abea None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:51.232846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3236e5-1f0f-4c57-978a-c4460f934181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:51.233698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3236e5-1f0f-4c57-978a-c4460f934181 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:51.234068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3236e5-1f0f-4c57-978a-c4460f934181 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:51.234470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3236e5-1f0f-4c57-978a-c4460f934181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:51.244178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3236e5-1f0f-4c57-978a-c4460f934181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:51.245687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3236e5-1f0f-4c57-978a-c4460f934181 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:51.255218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 766/1546] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:31:52.858056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6d2b-fe3a-4e9a-bc79-2ec5c653a650 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:52.858641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6d2b-fe3a-4e9a-bc79-2ec5c653a650 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:52.858641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e6d2b-fe3a-4e9a-bc79-2ec5c653a650 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:52.860113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 781/1547] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:31:52.895445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:52.895445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:52.895881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:52.931551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:31:52.933622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:53.072367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 767/1548] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:31:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:31:53.086969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6d2b-fe3a-4e9a-bc79-2ec5c653a650 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:31:53.090342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a38e6d2b-fe3a-4e9a-bc79-2ec5c653a650 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:31:53.125190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e6d2b-fe3a-4e9a-bc79-2ec5c653a650 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:31:53.136477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c3b83-17ee-475d-bcd3-8756d883cb3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:31:53.138403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c3b83-17ee-475d-bcd3-8756d883cb3a service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:31:53.138758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c3b83-17ee-475d-bcd3-8756d883cb3a service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:31:53.139249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c3b83-17ee-475d-bcd3-8756d883cb3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:31:53.157339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c3b83-17ee-475d-bcd3-8756d883cb3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:31:53.157339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c3b83-17ee-475d-bcd3-8756d883cb3a service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:31:53.177593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 782/1549] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:31:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:00.643728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:00.645985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:00.762832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:00.768362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c260624-2519-4639-9bf3-044a1be42539 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:00.773970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:00.774879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:00.775374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:00.775809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] PATH_INFO: `/v3/users/18cf47c3c9314623ad5956e95454eee6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:00.777143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18cf47c3c9314623ad5956e95454eee6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:00.779396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:00.869550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18cf47c3c9314623ad5956e95454eee6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:00.871060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18cf47c3c9314623ad5956e95454eee6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:00.871753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18cf47c3c9314623ad5956e95454eee6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:00.879708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18cf47c3c9314623ad5956e95454eee6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:00.889040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18cf47c3c9314623ad5956e95454eee6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:00.907211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 768/1550] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:00 2022] DELETE /identity/v3/users/18cf47c3c9314623ad5956e95454eee6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:00.917621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:00.917621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:00.917621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:00.931769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:00.933560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:00.937569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:00.994068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 783/1551] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:01.004195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:01.005616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:01.063890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433d574-64e4-460e-be5d-348638018e21 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:01.067471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:01.068070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:01.068350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:01.068622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:01.078553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:01.079973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:01.089283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 769/1552] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:01.364464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:01.366026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:01.379038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd16ca5-730a-42a8-8c54-be08f70ed5d3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:01.385536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:01.385536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:01.385536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:01.385536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] PATH_INFO: `/v3/projects/e62f3c60fa5d4bd283ada2dacb3d73a5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:01.387228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e62f3c60fa5d4bd283ada2dacb3d73a5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:01.389210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:01.440328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e62f3c60fa5d4bd283ada2dacb3d73a5', 'request_id': 'req-82118f06-5ef5-4be2-8256-c27a6360462d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.446751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e62f3c60fa5d4bd283ada2dacb3d73a5', 'request_id': 'req-82118f06-5ef5-4be2-8256-c27a6360462d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.455616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e62f3c60fa5d4bd283ada2dacb3d73a5', 'request_id': 'req-82118f06-5ef5-4be2-8256-c27a6360462d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.467007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] The token cache is being invalidate because project e62f3c60fa5d4bd283ada2dacb3d73a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:01.467292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.467958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.472378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 784/1553] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:01 2022] DELETE /identity/v3/projects/e62f3c60fa5d4bd283ada2dacb3d73a5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:01.473621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:01.484791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f43112-0c51-419d-8f4c-3c9e2344e8d5 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:01.542793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:01.543477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:01.543756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:01.544029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] PATH_INFO: `/v3/users/385e43e4979543828b940e6d71362992` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:01.545741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=385e43e4979543828b940e6d71362992)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:01.547662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:01.570366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:01.595252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '385e43e4979543828b940e6d71362992'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.595840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '385e43e4979543828b940e6d71362992'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.596499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '385e43e4979543828b940e6d71362992'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.603241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '385e43e4979543828b940e6d71362992'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.611074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6fe6e952-7448-4194-bcf0-3f77bfef10a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '385e43e4979543828b940e6d71362992'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:01.628777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 770/1554] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:01 2022] DELETE /identity/v3/users/385e43e4979543828b940e6d71362992 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:01.982341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:01.984052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.018452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82118f06-5ef5-4be2-8256-c27a6360462d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.099488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.100189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.100470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.100742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] PATH_INFO: `/v3/projects/d63c543aadfd421cbf280bda36d7e2e0` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.101946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d63c543aadfd421cbf280bda36d7e2e0)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.103834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.143633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:02.153783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd63c543aadfd421cbf280bda36d7e2e0', 'request_id': 'req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:02.159941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd63c543aadfd421cbf280bda36d7e2e0', 'request_id': 'req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:02.167477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd63c543aadfd421cbf280bda36d7e2e0', 'request_id': 'req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:02.177906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] The token cache is being invalidate because project d63c543aadfd421cbf280bda36d7e2e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:02.178191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:02.178834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4ced6f1f-28b2-4b70-9eb6-81c26e097a22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:02.180341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 785/1555] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:01 2022] DELETE /identity/v3/projects/d63c543aadfd421cbf280bda36d7e2e0 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:02.185819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.186365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.186451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.205680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:02.206881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.265431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 771/1556] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:02.273255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.273573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.273852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.294781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:02.295993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.303802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 786/1557] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:02.308057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.317699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f02063-8429-444b-ba4d-6113a7c76a85 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.322349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.323039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.323325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.323597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.324232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:02.324531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.326185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.337546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 772/1558] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:02.342429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.351571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0179a86-f7e2-4af5-8ff3-fb9c96c9bbca None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.356970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.357600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.357883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.358164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.358790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.360694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.387109 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 787/1559] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:02.391267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.401980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82290674-c314-4554-ac4f-167c19efd12b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.461599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.462202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.462479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.462749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.463375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.465835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.486167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 773/1560] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:02.490428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.500943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0943a-78f4-4510-9780-550f7c08b6a3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.505552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.506143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.506428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.506702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.507340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.508934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.519008 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 788/1561] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:02.523226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.534370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89d3e0-3676-4ee4-b5f8-9944d644f153 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.538886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec66c9-5d76-4f7b-b642-7a883d56ec81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.539478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec66c9-5d76-4f7b-b642-7a883d56ec81 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.539753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec66c9-5d76-4f7b-b642-7a883d56ec81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.540019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec66c9-5d76-4f7b-b642-7a883d56ec81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.540647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec66c9-5d76-4f7b-b642-7a883d56ec81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.543315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec66c9-5d76-4f7b-b642-7a883d56ec81 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.553697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 774/1562] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:02.557747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.568042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6475c09d-5bcc-4646-9f33-f7618bd237e2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.572592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.573230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.573735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.574101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] PATH_INFO: `/v3/projects/94272e3eda91497fa6c3c82b7fe72429/users/d32f8b415d8943548a2f31c6fb15f78f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.589790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94272e3eda91497fa6c3c82b7fe72429, user_id=d32f8b415d8943548a2f31c6fb15f78f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.592519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.607095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 789/1563] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:02 2022] PUT /identity/v3/projects/94272e3eda91497fa6c3c82b7fe72429/users/d32f8b415d8943548a2f31c6fb15f78f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:02.617080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.617463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.617741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.626607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:02.648924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] MFA Rules not processed for user `d32f8b415d8943548a2f31c6fb15f78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:02.650115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:02.715934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 775/1564] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:02.720397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.781747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8682a2e-4b55-4027-bda8-54490a3625e7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.786254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.786846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.787128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.787400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.788015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.789958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.816105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 790/1565] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/projects => generated 369 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:02.820297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.831443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d244afe7-155e-45db-a7c5-a6680958be84 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.887588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.888189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.888470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.888739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.889396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.891308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.913204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 776/1566] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:02 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:02.917135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.927706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ac31b-60a8-425c-a64c-1c271416635b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.932284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.932879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.933161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.933488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.934127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.935726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.947059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 791/1567] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:02.951840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:02.961829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c53d90-018e-4029-9cf4-deabb62360bb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:02.967362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:02.968001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:02.968283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:02.968556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] PATH_INFO: `/v3/projects/99a326afdc21459f85f95a10da915cd6/users/ad5a80fbc232495d88399c7e9c4d2bff/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:02.983016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a326afdc21459f85f95a10da915cd6, user_id=ad5a80fbc232495d88399c7e9c4d2bff, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:02.985716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:02.999324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 777/1568] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:02 2022] PUT /identity/v3/projects/99a326afdc21459f85f95a10da915cd6/users/ad5a80fbc232495d88399c7e9c4d2bff/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:03.003283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:03.013928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42121f95-e965-460d-9568-9e04b9dd92ed admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:03.050807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:03.089700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686f2c9-b4af-4763-8134-871822cd0f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:03.090364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f2c9-b4af-4763-8134-871822cd0f83 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:03.090645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f2c9-b4af-4763-8134-871822cd0f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:03.090919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686f2c9-b4af-4763-8134-871822cd0f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:03.091575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686f2c9-b4af-4763-8134-871822cd0f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:03.093345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686f2c9-b4af-4763-8134-871822cd0f83 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:03.110279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:03.116158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 792/1569] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:03.122605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ba65c-e5b7-42b7-8bc6-c824e33beb83 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:03.127557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:03.128225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:03.128508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:03.128895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] PATH_INFO: `/v3/projects/99a326afdc21459f85f95a10da915cd6/users/ad5a80fbc232495d88399c7e9c4d2bff/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:03.131395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99a326afdc21459f85f95a10da915cd6, user_id=ad5a80fbc232495d88399c7e9c4d2bff, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:03.134102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:03.146423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 778/1570] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:03 2022] PUT /identity/v3/projects/99a326afdc21459f85f95a10da915cd6/users/ad5a80fbc232495d88399c7e9c4d2bff/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:03.158442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:03.158849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:03.159280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:03.189829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] MFA Rules not processed for user `ad5a80fbc232495d88399c7e9c4d2bff`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:03.191053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:03.284886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 793/1571] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:03 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:03.296365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:03.298137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:03.351601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:03.358301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fa701-7544-43a7-a52d-258ec0e22986 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:03.361992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0230689-dde2-413f-9938-786f60f7f63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:03.362595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0230689-dde2-413f-9938-786f60f7f63e service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:03.362874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0230689-dde2-413f-9938-786f60f7f63e service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:03.363146 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0230689-dde2-413f-9938-786f60f7f63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:03.372797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0230689-dde2-413f-9938-786f60f7f63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:03.374245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0230689-dde2-413f-9938-786f60f7f63e service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:03.383565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 779/1572] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:03 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:04.796883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:04.857684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b0683-bb2b-4bb8-a215-85c8370f3d62 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:04.862085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:04.862701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:04.862998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:04.863267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] PATH_INFO: `/v3/users/d32f8b415d8943548a2f31c6fb15f78f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:04.864363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32f8b415d8943548a2f31c6fb15f78f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:04.866307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:04.912266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32f8b415d8943548a2f31c6fb15f78f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:04.919534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32f8b415d8943548a2f31c6fb15f78f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:04.920152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32f8b415d8943548a2f31c6fb15f78f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:04.922513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:04.929187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32f8b415d8943548a2f31c6fb15f78f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:04.929894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32f8b415d8943548a2f31c6fb15f78f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:04.953173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 794/1573] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:04 2022] DELETE /identity/v3/users/d32f8b415d8943548a2f31c6fb15f78f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:04.957884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:04.958403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:04.958579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:04.977935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:04.979134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:05.038795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 780/1574] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:05.050002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:05.051427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:05.111168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd00c00-dade-439a-af19-e108d0731ea6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:05.114887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:05.115499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:05.115873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:05.116151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:05.126021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:05.127461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:05.137647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 795/1575] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:05.435027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:05.436505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:05.450463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ce973-1c60-42e8-a071-2cbab9e1f98d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:05.455661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:05.456512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:05.456748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:05.457047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] PATH_INFO: `/v3/projects/94272e3eda91497fa6c3c82b7fe72429` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:05.475626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94272e3eda91497fa6c3c82b7fe72429)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:05.477913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:05.537758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94272e3eda91497fa6c3c82b7fe72429', 'request_id': 'req-27552e49-693e-4c52-8b18-8759c4787270'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.546771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94272e3eda91497fa6c3c82b7fe72429', 'request_id': 'req-27552e49-693e-4c52-8b18-8759c4787270'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.554013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94272e3eda91497fa6c3c82b7fe72429', 'request_id': 'req-27552e49-693e-4c52-8b18-8759c4787270'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.575390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] The token cache is being invalidate because project 94272e3eda91497fa6c3c82b7fe72429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:05.575899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.576807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.579743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 781/1576] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:05 2022] DELETE /identity/v3/projects/94272e3eda91497fa6c3c82b7fe72429 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:05.583406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:05.599181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c8946f-8bba-496d-9785-4ec20252c12a service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:05.695673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:05.696312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:05.696588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:05.696860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] PATH_INFO: `/v3/users/ad5a80fbc232495d88399c7e9c4d2bff` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:05.698030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5a80fbc232495d88399c7e9c4d2bff)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:05.700140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:05.756047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5a80fbc232495d88399c7e9c4d2bff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.764078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5a80fbc232495d88399c7e9c4d2bff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.764829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5a80fbc232495d88399c7e9c4d2bff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.774787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5a80fbc232495d88399c7e9c4d2bff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.775452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8952009a-b4de-42c7-a19a-deec3da28061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5a80fbc232495d88399c7e9c4d2bff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:05.787349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:05.795773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 796/1577] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:05 2022] DELETE /identity/v3/users/ad5a80fbc232495d88399c7e9c4d2bff => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:06.084902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.086489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:06.101120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27552e49-693e-4c52-8b18-8759c4787270 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.168738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.169627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.169907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.170179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] PATH_INFO: `/v3/projects/99a326afdc21459f85f95a10da915cd6` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.171508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99a326afdc21459f85f95a10da915cd6)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.173482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.191617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:06.225940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99a326afdc21459f85f95a10da915cd6', 'request_id': 'req-e79b8532-a75f-4e47-a9df-b49d5383b174'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:06.234947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99a326afdc21459f85f95a10da915cd6', 'request_id': 'req-e79b8532-a75f-4e47-a9df-b49d5383b174'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:06.244068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99a326afdc21459f85f95a10da915cd6', 'request_id': 'req-e79b8532-a75f-4e47-a9df-b49d5383b174'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:06.257380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] The token cache is being invalidate because project 99a326afdc21459f85f95a10da915cd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:06.257844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:06.258524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e79b8532-a75f-4e47-a9df-b49d5383b174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:06.260126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 782/1578] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:06 2022] DELETE /identity/v3/projects/99a326afdc21459f85f95a10da915cd6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:06.266196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.266664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.266923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.287931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:06.289236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:06.357732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 797/1579] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:06.364946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.365232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.365538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.389745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:06.391023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:06.399431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 783/1580] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:06.403814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.415870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec37d33-2471-4ab8-bcc3-466fc31ebe3f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.422125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.422990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.423425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.423861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.424904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:06.426230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.427952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.445950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 798/1581] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:06.447639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.461083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381b063-4cf5-496c-a3fd-e564bb9adc4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.467288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.467508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.467928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.468321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.468958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.471273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.505668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 784/1582] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:06 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:06.510794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.522709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fe3fc-268d-4c6e-8e14-23b860ddb0f4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.640862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.641758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.642105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.642444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.643297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.645353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.694834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 799/1583] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:06 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:06.700488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.724807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1e67-4d49-463d-b0e0-a3197f2c8ac1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.738529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.739015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.739768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.741775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.742690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.747493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.762067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 785/1584] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:06.765900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.781886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4c7b32-89de-4472-aa2c-c281219056c3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.796215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337c5cf-effd-4c1e-88d9-a94a108b542c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.797384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c5cf-effd-4c1e-88d9-a94a108b542c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.797671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c5cf-effd-4c1e-88d9-a94a108b542c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.797991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337c5cf-effd-4c1e-88d9-a94a108b542c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.798940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c5cf-effd-4c1e-88d9-a94a108b542c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.800472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337c5cf-effd-4c1e-88d9-a94a108b542c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.818405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 800/1585] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:06 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:06.820409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:06.834853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5931148-3714-4688-bd29-771f1c9a1c57 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:06.845437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:06.846991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.847686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.848493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] PATH_INFO: `/v3/projects/ced23104aed7402e8238bbd4172d41c2/users/00e1cb9fdf8141e3bca2a15794bf68b5/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.879578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ced23104aed7402e8238bbd4172d41c2, user_id=00e1cb9fdf8141e3bca2a15794bf68b5, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:06.882610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:06.914086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 786/1586] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:06 2022] PUT /identity/v3/projects/ced23104aed7402e8238bbd4172d41c2/users/00e1cb9fdf8141e3bca2a15794bf68b5/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:06.919203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:06.919772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:06.919947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:06.968793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] MFA Rules not processed for user `00e1cb9fdf8141e3bca2a15794bf68b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:06.970387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:07.125581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:07.161976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 801/1587] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:06 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:07.167421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:07.366378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d48fdf-7563-454b-bedb-a2eb266bd50e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:07.378008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:07.378749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:07.379105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:07.379510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:07.380284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:07.382797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:07.422911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 787/1588] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:07 2022] POST /identity/v3/projects => generated 381 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:07.426949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:07.464093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a180eb-79c0-474b-8802-2975e452d4f1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:07.626055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:07.627900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:07.628382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:07.628740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:07.629509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:07.631705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:07.677670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 802/1589] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:07 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:07.682032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:07.697375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad44c7-58c4-417c-aa31-bab5db6a302d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:07.714235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:07.716479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:07.717164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:07.718606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:07.720297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:07.723326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:07.744175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 788/1590] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:07.749070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:07.766231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592dae9-1359-4362-81cc-3d20117c8ccf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:07.771022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:07.771621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:07.771901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:07.772180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] PATH_INFO: `/v3/projects/f96761d2fba249f08bca1f514a4c58c2/users/4072e1c0a4db4394a26760746e1be3cf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:07.806891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96761d2fba249f08bca1f514a4c58c2, user_id=4072e1c0a4db4394a26760746e1be3cf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:07.810789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:07.827254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 803/1591] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:07 2022] PUT /identity/v3/projects/f96761d2fba249f08bca1f514a4c58c2/users/4072e1c0a4db4394a26760746e1be3cf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:07.831206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:07.842652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753276e-7a05-4ac4-997a-d6149a3b65f6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:07.853261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:07.966352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5468b288-f452-419c-98d6-d66e1bb74f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:07.967198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b288-f452-419c-98d6-d66e1bb74f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:07.967641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b288-f452-419c-98d6-d66e1bb74f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:07.967994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5468b288-f452-419c-98d6-d66e1bb74f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:07.971509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468b288-f452-419c-98d6-d66e1bb74f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:07.973512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5468b288-f452-419c-98d6-d66e1bb74f05 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:07.986520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 789/1592] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:07.991088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:08.004604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40e7ac8-c624-4ee1-bd7a-55bb2189a85a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:08.011208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:08.012354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.012635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.012908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] PATH_INFO: `/v3/projects/f96761d2fba249f08bca1f514a4c58c2/users/4072e1c0a4db4394a26760746e1be3cf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.015762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f96761d2fba249f08bca1f514a4c58c2, user_id=4072e1c0a4db4394a26760746e1be3cf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:08.018612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:08.030660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 804/1593] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:07 2022] PUT /identity/v3/projects/f96761d2fba249f08bca1f514a4c58c2/users/4072e1c0a4db4394a26760746e1be3cf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:08.043116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.045272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.045272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.118577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] MFA Rules not processed for user `4072e1c0a4db4394a26760746e1be3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:08.129242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.221210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 790/1594] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:08.233217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:08.235095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.350053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cadb3-eea9-4f77-ad2b-32b5a557255e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:08.353913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc1d33-e192-4be3-ad28-876fa2e12dc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:08.354564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc1d33-e192-4be3-ad28-876fa2e12dc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.354787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc1d33-e192-4be3-ad28-876fa2e12dc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.355058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc1d33-e192-4be3-ad28-876fa2e12dc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.365990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc1d33-e192-4be3-ad28-876fa2e12dc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:08.367442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc1d33-e192-4be3-ad28-876fa2e12dc6 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:08.378864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 805/1595] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:08.413400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:08.489115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6da8914-bfbb-47b5-9af4-37339dc473ef None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:08.494504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:08.495305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.495596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.495863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] PATH_INFO: `/v3/users/00e1cb9fdf8141e3bca2a15794bf68b5` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.497604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e1cb9fdf8141e3bca2a15794bf68b5)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:08.499518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:08.537338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:08.547272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e1cb9fdf8141e3bca2a15794bf68b5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:08.547975 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e1cb9fdf8141e3bca2a15794bf68b5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:08.548656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e1cb9fdf8141e3bca2a15794bf68b5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:08.556138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e1cb9fdf8141e3bca2a15794bf68b5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:08.564503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e1cb9fdf8141e3bca2a15794bf68b5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:08.582354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 791/1596] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:08 2022] DELETE /identity/v3/users/00e1cb9fdf8141e3bca2a15794bf68b5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:08.586818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.587292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.587383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.607911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:08.608996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.682114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:08.709131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 806/1597] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:08.722772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:08.724990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:08.878650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a144650-a03a-4c52-9b5a-019e195dcf7e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:08.883605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:08.884432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:08.884811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:08.885333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:08.899522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:08.901193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:08.913720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 792/1598] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:32:09.351338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:09.353540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:09.376302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c647f6cb-fbf7-45c3-bda0-9d20ffd44e9c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:09.388867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:09.389554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:09.389837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:09.390106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] PATH_INFO: `/v3/projects/ced23104aed7402e8238bbd4172d41c2` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:09.392938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ced23104aed7402e8238bbd4172d41c2)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:09.395163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:09.487044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ced23104aed7402e8238bbd4172d41c2', 'request_id': 'req-a1408ba8-76f4-498a-846d-5912aed9f15d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.500973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ced23104aed7402e8238bbd4172d41c2', 'request_id': 'req-a1408ba8-76f4-498a-846d-5912aed9f15d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.516078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ced23104aed7402e8238bbd4172d41c2', 'request_id': 'req-a1408ba8-76f4-498a-846d-5912aed9f15d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.531790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] The token cache is being invalidate because project ced23104aed7402e8238bbd4172d41c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:09.532573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.533609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.539171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 807/1599] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:09 2022] DELETE /identity/v3/projects/ced23104aed7402e8238bbd4172d41c2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:09.543638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:09.560350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb5e46-eb0b-45bc-84c8-3fd78b9398cb service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:09.580875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:09.582920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:09.651665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:09.652301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:09.652579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:09.652966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] PATH_INFO: `/v3/users/4072e1c0a4db4394a26760746e1be3cf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:09.654388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4072e1c0a4db4394a26760746e1be3cf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:09.657683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:09.718583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1408ba8-76f4-498a-846d-5912aed9f15d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:09.724039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:09.724806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:09.725152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:09.725518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] PATH_INFO: `/v3/users/95625040769c4a4582ae52117ee1a1fd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:09.727263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95625040769c4a4582ae52117ee1a1fd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:09.729737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:09.734537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4072e1c0a4db4394a26760746e1be3cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.735252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4072e1c0a4db4394a26760746e1be3cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.738120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4072e1c0a4db4394a26760746e1be3cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.748377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4072e1c0a4db4394a26760746e1be3cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.753860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:09.774098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-045b0abc-e399-4648-8019-65770ad2d554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4072e1c0a4db4394a26760746e1be3cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.823612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95625040769c4a4582ae52117ee1a1fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.843319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95625040769c4a4582ae52117ee1a1fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.844547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95625040769c4a4582ae52117ee1a1fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.861508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95625040769c4a4582ae52117ee1a1fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.864139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95625040769c4a4582ae52117ee1a1fd'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:09.875402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:09.880764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 793/1600] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:09 2022] DELETE /identity/v3/users/4072e1c0a4db4394a26760746e1be3cf => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:09.898158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 808/1601] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:09 2022] DELETE /identity/v3/users/95625040769c4a4582ae52117ee1a1fd => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:09.900168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:09.901045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:09.901045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:09.948056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:09.951753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:10.078721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 794/1602] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:10.098303 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:10.114110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:10.266450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce934d-80cd-4adc-95f1-d0fb2bb9cc65 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:10.271621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ef76c-a308-44bf-888a-60cdf6cb30fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:10.272321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef76c-a308-44bf-888a-60cdf6cb30fd service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:10.272533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef76c-a308-44bf-888a-60cdf6cb30fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:10.272930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ef76c-a308-44bf-888a-60cdf6cb30fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:10.290784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef76c-a308-44bf-888a-60cdf6cb30fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:10.292622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ef76c-a308-44bf-888a-60cdf6cb30fd service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:10.316682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 809/1603] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:10.586948 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:10.588495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:10.640606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedf948b-12f6-47c5-8374-88a766b18e1d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:10.647389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:10.648259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:10.648947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:10.648947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] PATH_INFO: `/v3/projects/f96761d2fba249f08bca1f514a4c58c2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:10.650347 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f96761d2fba249f08bca1f514a4c58c2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:10.653436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:10.717459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f96761d2fba249f08bca1f514a4c58c2', 'request_id': 'req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:10.729495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f96761d2fba249f08bca1f514a4c58c2', 'request_id': 'req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:10.741292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f96761d2fba249f08bca1f514a4c58c2', 'request_id': 'req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:10.758125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] The token cache is being invalidate because project f96761d2fba249f08bca1f514a4c58c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:10.758487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:10.759381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:10.761409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 795/1604] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:10 2022] DELETE /identity/v3/projects/f96761d2fba249f08bca1f514a4c58c2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:10.767447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f16c9d-5b33-4bca-b093-6b7c7178f02b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:10.767962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f16c9d-5b33-4bca-b093-6b7c7178f02b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:10.768054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f16c9d-5b33-4bca-b093-6b7c7178f02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:10.808734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c6f16c9d-5b33-4bca-b093-6b7c7178f02b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:10.811361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6f16c9d-5b33-4bca-b093-6b7c7178f02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:10.826538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:10.829499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:10.909343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 810/1605] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:10.930243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:10.930530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:10.930858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:10.943169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d47f11-dd32-4f9b-aef3-49a276d42f2e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:10.948454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:10.949940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:10.950179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:10.950374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] PATH_INFO: `/v3/projects/92938ce5f9434f6796e067d05ca6a2e9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:10.952457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92938ce5f9434f6796e067d05ca6a2e9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:10.954743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:10.955040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:10.956642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:10.957541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:10.969600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 811/1606] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:10.973938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:10.987284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499a048-6fea-4f93-b6da-4383ff9789ae None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:10.997139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:10.997813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:10.998098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:10.998373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:10.999047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:10.999354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.001061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.028270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 812/1607] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:11.032424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.057348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7db3-91fc-41e9-a7aa-4aa14c87e664 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.075795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92938ce5f9434f6796e067d05ca6a2e9', 'request_id': 'req-b62534e3-db78-4d62-833e-4bc978b2fba1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.094382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92938ce5f9434f6796e067d05ca6a2e9', 'request_id': 'req-b62534e3-db78-4d62-833e-4bc978b2fba1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.110438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92938ce5f9434f6796e067d05ca6a2e9', 'request_id': 'req-b62534e3-db78-4d62-833e-4bc978b2fba1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.137824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] The token cache is being invalidate because project 92938ce5f9434f6796e067d05ca6a2e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:11.138131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.144237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.152343 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 796/1608] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:10 2022] DELETE /identity/v3/projects/92938ce5f9434f6796e067d05ca6a2e9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:11.153608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.158455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.190537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.190881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.191169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.191451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.192330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.198401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.229191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:11.249414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 813/1609] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:11 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:11.250346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.254067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.264171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62534e3-db78-4d62-833e-4bc978b2fba1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.365372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.366033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.366430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.366580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] PATH_INFO: `/v3/users/b6852a16561048beba96e8cc899df6a7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.367821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6852a16561048beba96e8cc899df6a7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.370070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.404445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c9d06b-a9d2-4829-b872-bf3c6a0c4b21 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.419338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.420632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.420946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.421250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] PATH_INFO: `/v3/users/6a36e646bce441338698dd64c1050d55` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.423674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a36e646bce441338698dd64c1050d55)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.426657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.434837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:11.446372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6852a16561048beba96e8cc899df6a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.447284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6852a16561048beba96e8cc899df6a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.447911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6852a16561048beba96e8cc899df6a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.461664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6852a16561048beba96e8cc899df6a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.476280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6852a16561048beba96e8cc899df6a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.515529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 797/1610] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:11 2022] DELETE /identity/v3/users/b6852a16561048beba96e8cc899df6a7 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:11.516473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.518033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.523189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a36e646bce441338698dd64c1050d55'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.533291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a36e646bce441338698dd64c1050d55'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.534123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a36e646bce441338698dd64c1050d55'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.546947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a36e646bce441338698dd64c1050d55'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.547720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d32b01c8-eab9-4147-9a4a-89b2393a24bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a36e646bce441338698dd64c1050d55'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:11.586101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 814/1611] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:11 2022] DELETE /identity/v3/users/6a36e646bce441338698dd64c1050d55 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:11.587285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.587567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.587838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.614288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:11.615503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.644767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fbe477-8d03-4574-bf5b-972a810a1610 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.696744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 815/1612] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:11.710967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.712649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.744768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:11.796448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.797724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.798453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.798917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.801935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.804323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.829614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b95e09-3c88-48ec-9266-b773621f783d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.838989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.839684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.840152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.840638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.847277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 798/1613] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:11 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:11.851102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.853458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.853605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:11.855345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.867797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea2dfa-f8d7-47d4-8850-a4f315e74acf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.868734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 816/1614] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:32:11.874874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.875681 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.876100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.876513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.877299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.879117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.890492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 799/1615] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:11.894534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.905642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7197e5-9573-455f-b45a-82528b59d057 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.911192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8e4cd-abe1-4aa5-a77d-8a8bcf159f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.912055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8e4cd-abe1-4aa5-a77d-8a8bcf159f11 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.912441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8e4cd-abe1-4aa5-a77d-8a8bcf159f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.912996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8e4cd-abe1-4aa5-a77d-8a8bcf159f11 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:11.914827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8e4cd-abe1-4aa5-a77d-8a8bcf159f11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:11.917223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8e4cd-abe1-4aa5-a77d-8a8bcf159f11 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:11.941634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 817/1616] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:11.946541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:11.967643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd6749-4366-4556-9f27-2304273bd49c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:11.981878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:11.982333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:11.982704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:11.982890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] PATH_INFO: `/v3/projects/5ff0ab9f8f224120867b32a3e5596562/users/d7c71fee306a494c9caba3dc0869e816/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.001398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff0ab9f8f224120867b32a3e5596562, user_id=d7c71fee306a494c9caba3dc0869e816, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.004725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.020544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 800/1617] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:11 2022] PUT /identity/v3/projects/5ff0ab9f8f224120867b32a3e5596562/users/d7c71fee306a494c9caba3dc0869e816/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:12.034269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.035150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.035630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.083645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] MFA Rules not processed for user `d7c71fee306a494c9caba3dc0869e816`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:12.085584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.172982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 818/1618] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:12.182934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.223698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:12.237384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.239199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.320576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f65a2d-808f-4c5e-ae84-7b5d6c00db7a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.329395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.330423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.330909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.331408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.332250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.334548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.343172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416cf1a-2dcb-4682-b398-6cbbf61df897 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.348159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.348798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.349121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.349430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] PATH_INFO: `/v3/projects/aa8fc3a3d1a44281a4ff7acdba1e6787` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.350729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8fc3a3d1a44281a4ff7acdba1e6787)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.352772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.384120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 801/1619] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:12 2022] POST /identity/v3/projects => generated 373 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:12.388847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.413339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:12.424607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8fc3a3d1a44281a4ff7acdba1e6787', 'request_id': 'req-a4093504-f84a-4254-89a4-42ae9a45f514'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.432872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8fc3a3d1a44281a4ff7acdba1e6787', 'request_id': 'req-a4093504-f84a-4254-89a4-42ae9a45f514'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.445255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8fc3a3d1a44281a4ff7acdba1e6787', 'request_id': 'req-a4093504-f84a-4254-89a4-42ae9a45f514'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.470357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] The token cache is being invalidate because project aa8fc3a3d1a44281a4ff7acdba1e6787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:12.470691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.471802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.477109 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 819/1620] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:12 2022] DELETE /identity/v3/projects/aa8fc3a3d1a44281a4ff7acdba1e6787 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:12.478219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.479745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.491907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2feea2-a4a5-472b-817d-c5f56227e55e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.560000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f3da3-a3ce-453c-b682-087ead663c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.560615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3da3-a3ce-453c-b682-087ead663c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.560899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3da3-a3ce-453c-b682-087ead663c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.561169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f3da3-a3ce-453c-b682-087ead663c3a admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.561832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f3da3-a3ce-453c-b682-087ead663c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.563828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f3da3-a3ce-453c-b682-087ead663c3a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.572106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4093504-f84a-4254-89a4-42ae9a45f514 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.577234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.577912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.578354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.578655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] PATH_INFO: `/v3/projects/a0e18d53c13f46bf8dbb5ff9588f6366` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.579925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0e18d53c13f46bf8dbb5ff9588f6366)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.582601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.587613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 802/1621] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:12 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:12.589067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.589376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.589657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.614963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:12.616530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.625611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 803/1622] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:12.626707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.633117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.645057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0e18d53c13f46bf8dbb5ff9588f6366', 'request_id': 'req-a04543e4-841b-4410-ada9-59bbb33a65f8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.645537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7d981-6467-447e-9b25-155c78648853 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.652596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0e18d53c13f46bf8dbb5ff9588f6366', 'request_id': 'req-a04543e4-841b-4410-ada9-59bbb33a65f8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.663486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0e18d53c13f46bf8dbb5ff9588f6366', 'request_id': 'req-a04543e4-841b-4410-ada9-59bbb33a65f8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.680094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] The token cache is being invalidate because project a0e18d53c13f46bf8dbb5ff9588f6366 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:12.680420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.681090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a04543e4-841b-4410-ada9-59bbb33a65f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.682799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 820/1623] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:12 2022] DELETE /identity/v3/projects/a0e18d53c13f46bf8dbb5ff9588f6366 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:12.683275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:12.683881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.684767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.685082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.704814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:12.706050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.723657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.724496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.724877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.725248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.726037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.727712 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.740444 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 804/1624] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:12.742435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.745974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.763205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796c62a1-f80b-48fb-ba81-a4aebadbcc4b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.767872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.768522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.768785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.769063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] PATH_INFO: `/v3/users/b35522c5faff4b80a80421fd2a38486f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.770307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35522c5faff4b80a80421fd2a38486f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.772242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.775622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:12.786761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 821/1625] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:12.787660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.791864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.807215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdafe8-1f1d-4310-ad93-765d6356678c None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.814044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.815194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.815618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.816083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.817224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.819238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.830820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 822/1626] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:12.831559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.833354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35522c5faff4b80a80421fd2a38486f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.834061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35522c5faff4b80a80421fd2a38486f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.837552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35522c5faff4b80a80421fd2a38486f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.843470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9aa2-4797-4661-97ea-712007e3972b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.845836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35522c5faff4b80a80421fd2a38486f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.849476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.850139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.850458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.850753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.851418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:12.851764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.853450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.854054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35522c5faff4b80a80421fd2a38486f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:12.873090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 823/1627] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:12.873090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.874886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.875027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 805/1628] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:12 2022] DELETE /identity/v3/users/b35522c5faff4b80a80421fd2a38486f => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:12.876027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:12.877448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:12.901531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f64b4d-6775-4773-89a6-1c6d5beb6499 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.954062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ea711-746d-4dd0-aad8-9bdc22afab49 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:12.961165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.961911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.962385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.962813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:12.963455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:12.966418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:12.989669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8b3cb-de2b-4eaa-805b-c8203e4e6861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:12.992298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b3cb-de2b-4eaa-805b-c8203e4e6861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:12.992787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b3cb-de2b-4eaa-805b-c8203e4e6861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:12.993867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8b3cb-de2b-4eaa-805b-c8203e4e6861 admin admin] PATH_INFO: `/v3/projects/9bbdc6b8bc1f4201bbf43f0a8b47fca7/users/a027f75789b841989272652b315e5d14/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.003411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 806/1629] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:12 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:13.004497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.022307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8b3cb-de2b-4eaa-805b-c8203e4e6861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bbdc6b8bc1f4201bbf43f0a8b47fca7, user_id=a027f75789b841989272652b315e5d14, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.026705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8b3cb-de2b-4eaa-805b-c8203e4e6861 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.044947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 824/1630] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:12 2022] PUT /identity/v3/projects/9bbdc6b8bc1f4201bbf43f0a8b47fca7/users/a027f75789b841989272652b315e5d14/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:13.047616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:13.056232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.056411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.056898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.089725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] MFA Rules not processed for user `a027f75789b841989272652b315e5d14`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:13.091451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.098015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1270c-6995-4c55-9014-4a32638ba31f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.165912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.167038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.167325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.167597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.167773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 825/1631] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:13.171877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.172270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.173941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.183492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60624476-6992-49a3-b0c0-0df7410f6359 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.188228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.188939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.189352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.189725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.190405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.192524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.205809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 807/1632] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:13.207369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.209864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.225005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406a38d1-3ae5-44de-b013-55b1f30911a4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.233033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.233485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 826/1633] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:13.234005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.234365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.234475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.234866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] PATH_INFO: `/v3/projects/2673617b81a24d80b7335d1238a38c42` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.236131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2673617b81a24d80b7335d1238a38c42)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.238568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.283139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:13.300032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2673617b81a24d80b7335d1238a38c42', 'request_id': 'req-37e4b857-8589-43e7-89b7-215918115105'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:13.310313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2673617b81a24d80b7335d1238a38c42', 'request_id': 'req-37e4b857-8589-43e7-89b7-215918115105'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:13.318054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2673617b81a24d80b7335d1238a38c42', 'request_id': 'req-37e4b857-8589-43e7-89b7-215918115105'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:13.325502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954d9d0-0f70-4427-8f92-adafa900a8f8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.331139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] The token cache is being invalidate because project 2673617b81a24d80b7335d1238a38c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:13.331627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:13.332346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:13.337864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 808/1634] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:13 2022] DELETE /identity/v3/projects/2673617b81a24d80b7335d1238a38c42 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:13.337864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.339612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.394801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b2761c-1d79-4677-b256-f2adb3a345a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.395366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2761c-1d79-4677-b256-f2adb3a345a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.395645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2761c-1d79-4677-b256-f2adb3a345a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.395945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b2761c-1d79-4677-b256-f2adb3a345a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.396588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2761c-1d79-4677-b256-f2adb3a345a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.398249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b2761c-1d79-4677-b256-f2adb3a345a5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.398714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:13.413298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 827/1635] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:13.413298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.413298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.413298 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.416327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e4b857-8589-43e7-89b7-215918115105 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.421871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f593504-aace-483a-999f-f162f70abe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.422615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f593504-aace-483a-999f-f162f70abe66 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.422892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f593504-aace-483a-999f-f162f70abe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.423161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f593504-aace-483a-999f-f162f70abe66 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.423784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f593504-aace-483a-999f-f162f70abe66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.425767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f593504-aace-483a-999f-f162f70abe66 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.431170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:13.432394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.441199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 828/1636] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:13.442039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.443876 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.447895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 809/1637] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:13.449132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.449466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.450265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.456113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950ff34-092d-48c8-a8d7-c5fa9408c49a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.460650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.461238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.461542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.461809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.462437 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.464051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.470021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:13.471234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.474819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 829/1638] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:13.475691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.479494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 810/1639] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:13.480246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.485766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406eb273-7aa8-402d-a3b3-01b5080d62c7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.489749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8882b66-13b1-4bdb-8907-fd4346ec6ecb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.490531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.491126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.491406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.491678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.492312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.494187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.494811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.495445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.495727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.496009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] PATH_INFO: `/v3/projects/1f17e062659d48e184518156ba1ba766/users/d44ab34fe9824eb89f8cb5b68d38bd97/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.508157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 830/1640] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:13.508157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.510989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f17e062659d48e184518156ba1ba766, user_id=d44ab34fe9824eb89f8cb5b68d38bd97, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.514380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.516965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef3faf1-c583-4b15-9c12-c6008d942548 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.523092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.523461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.523744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.524020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.524664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:13.525035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.526792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956df1f-f2ad-4889-9dcd-be50a2755cfc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.530756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 811/1641] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:13 2022] PUT /identity/v3/projects/1f17e062659d48e184518156ba1ba766/users/d44ab34fe9824eb89f8cb5b68d38bd97/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:13.531564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.540472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 831/1642] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:13.541943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.542154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.542436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.577143 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] MFA Rules not processed for user `d44ab34fe9824eb89f8cb5b68d38bd97`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:13.578395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:13.624477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3ba3a-9db6-436d-b6ec-3df4ac46bbb1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.629688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.630259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.630648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.631022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] PATH_INFO: `/v3/projects/ec2dba916db34c3b987bfdda1bebf80e/users/cc4a9a08e0684056bb3bb21a08b79801/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.647614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2dba916db34c3b987bfdda1bebf80e, user_id=cc4a9a08e0684056bb3bb21a08b79801, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.650332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.662458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 832/1643] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:13.664843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.667831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 812/1644] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:13 2022] PUT /identity/v3/projects/ec2dba916db34c3b987bfdda1bebf80e/users/cc4a9a08e0684056bb3bb21a08b79801/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:13.668750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.676888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefce70-d1be-4ea2-8b7d-d08d1668786f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.694328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:13.739259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8b803-4398-4326-9b6f-8b4207295a6b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.744879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.745383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.745581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.745832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.746072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.746530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.746941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.747108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.747279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.747905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.748979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.749832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.782736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 833/1645] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:13.782736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.794012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 813/1646] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:13.795228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.820734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:13.869518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f3d18-056b-4a9e-9d3e-cd84aa7c9e7b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.898631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd5cb8e-59fc-4607-a75d-2f946e303f31 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.904692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.905549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.905919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.906374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.907141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.909294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.944647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 814/1647] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/users => generated 294 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:13.945921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:13.958838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fb3e21-8c5f-4918-9687-bd3727fa9598 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:13.964760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.965521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.966015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.966280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.966972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.968920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.984513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:13.985164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:13.986000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:13.986000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:13.986503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:13.988641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:13.995761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 815/1648] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:13 2022] POST /identity/v3/users => generated 299 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:14.000359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.004035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 834/1649] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:14.004035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.013690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d074d-fe07-49f0-ba34-0d5166b7e431 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.015612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e48e1f-8069-46be-b319-aa8d2c65c3f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.018907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.019519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.019830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.020105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.020835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.020956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.021510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.021791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.022062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.022476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.022693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.024407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.034896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 835/1650] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:14.035624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.039399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 816/1651] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:14.040300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.054143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f143087-7f59-4096-9294-f976fd39163b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.056455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35775d46-0294-4e8a-933a-a2efc866b800 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.062577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090075fc-a9f2-47e7-8bda-5370d5def57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.063315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.063779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090075fc-a9f2-47e7-8bda-5370d5def57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.063978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.064258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.064532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] PATH_INFO: `/v3/projects/ec2dba916db34c3b987bfdda1bebf80e/users/cc4a9a08e0684056bb3bb21a08b79801/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.065103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090075fc-a9f2-47e7-8bda-5370d5def57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.065621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090075fc-a9f2-47e7-8bda-5370d5def57a admin admin] PATH_INFO: `/v3/projects/5c1bed66c14045218e8cbb3687387c9e/users/99073d9acc2542ba9a0c5e8f56af25ce/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.066590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2dba916db34c3b987bfdda1bebf80e, user_id=cc4a9a08e0684056bb3bb21a08b79801, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.069620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.084528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 836/1652] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:14 2022] PUT /identity/v3/projects/ec2dba916db34c3b987bfdda1bebf80e/users/cc4a9a08e0684056bb3bb21a08b79801/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:14.085336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090075fc-a9f2-47e7-8bda-5370d5def57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c1bed66c14045218e8cbb3687387c9e, user_id=99073d9acc2542ba9a0c5e8f56af25ce, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.085454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.088466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090075fc-a9f2-47e7-8bda-5370d5def57a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.103128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 817/1653] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:14 2022] PUT /identity/v3/projects/5c1bed66c14045218e8cbb3687387c9e/users/99073d9acc2542ba9a0c5e8f56af25ce/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:14.106900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.106900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.106900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.136150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:14.142144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] MFA Rules not processed for user `cc4a9a08e0684056bb3bb21a08b79801`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:14.143544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.197992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae97e9cb-81fb-41b6-a5fb-46b931f495c8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.242812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 818/1654] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:14.245594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.298139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:14.313251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.313991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.314171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.314444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.315095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.316722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.329510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 837/1655] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:14.330379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.333235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb82a3-55d2-43eb-aff8-cab79daecc6d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.333872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.339395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.340622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.341103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.341465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.342182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.343784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.354448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 819/1656] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:14.355267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.366570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136a3ef-935d-418b-a07f-93b179cc3ee1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.372200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.372990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.373271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.373574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] PATH_INFO: `/v3/projects/3fd116eb68cd415d9b265fcc0600a0b0/users/7b0e50b51d954c76955f5c8275a0242c/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.390556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd116eb68cd415d9b265fcc0600a0b0, user_id=7b0e50b51d954c76955f5c8275a0242c, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.393492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.413213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 820/1657] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:14 2022] PUT /identity/v3/projects/3fd116eb68cd415d9b265fcc0600a0b0/users/7b0e50b51d954c76955f5c8275a0242c/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:14.425467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.425680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e98aae2-0c1f-40c1-9b60-f36421d62dfc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.486436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbb0e9-ca72-41a2-9eba-afcc524701f7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.488037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.488663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.488947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.489466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.491215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.491513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc5aa2-6ed8-4011-b9e0-f9d39c44358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.492263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc5aa2-6ed8-4011-b9e0-f9d39c44358b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.492554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc5aa2-6ed8-4011-b9e0-f9d39c44358b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.492830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc5aa2-6ed8-4011-b9e0-f9d39c44358b admin admin] PATH_INFO: `/v3/projects/5c1bed66c14045218e8cbb3687387c9e/users/99073d9acc2542ba9a0c5e8f56af25ce/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.495047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc5aa2-6ed8-4011-b9e0-f9d39c44358b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c1bed66c14045218e8cbb3687387c9e, user_id=99073d9acc2542ba9a0c5e8f56af25ce, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.499699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc5aa2-6ed8-4011-b9e0-f9d39c44358b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.509496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 821/1658] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:14 2022] PUT /identity/v3/projects/5c1bed66c14045218e8cbb3687387c9e/users/99073d9acc2542ba9a0c5e8f56af25ce/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:14.510802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.511086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.511359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.543086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] MFA Rules not processed for user `7b0e50b51d954c76955f5c8275a0242c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:14.544719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.549450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:14.591380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.593860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f641d73-c3db-47ff-be4d-b2d32f02fa0a service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.610563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 838/1659] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:14.612424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.612918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.613533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.632200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 822/1660] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:14.633346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.652851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] MFA Rules not processed for user `99073d9acc2542ba9a0c5e8f56af25ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:14.654809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.685499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:14.755326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958afa54-97f3-4963-9185-c0ae78f592bf None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.763156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.764122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.764545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.764878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.766082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.768528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.811378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 839/1661] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:14.825632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 823/1662] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:14 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:14.827148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.829604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.836985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:14.843886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081cf1-1a1e-4f3e-937b-93d2dc22c973 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.916746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.917451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.917670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.918006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.918688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:14.921636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:14.935474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5711be89-03af-4d87-9ff8-23d212eff8b1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.948468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:14.950037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:14.950709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:14.951404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:14.958940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 840/1663] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:14 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:14.962681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:14.974505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ede893-89f8-4aac-ba5c-ba9ab3b2ac9c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:14.999299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:15.001282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.003834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.004294 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.005141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.011559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.033369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 841/1664] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:15.037628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:15.058606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a46b0-3ca6-4b09-b0b2-d30aaa9e79b4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:15.062908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.080308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.092777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:15.099689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.100035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.100313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] PATH_INFO: `/v3/projects/e7bdd2652a1e4a8eb093a9ee2adc31d1/users/fb54b7eb952043b7813a6ee04881da65/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.126512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 824/1665] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:15.134669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bdd2652a1e4a8eb093a9ee2adc31d1, user_id=fb54b7eb952043b7813a6ee04881da65, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.141247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.169686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 842/1666] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:15 2022] PUT /identity/v3/projects/e7bdd2652a1e4a8eb093a9ee2adc31d1/users/fb54b7eb952043b7813a6ee04881da65/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:15.174242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:15.190322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2b188b-b8f9-4e2a-a82f-6fa2e99f483c service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:15.327377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:15.328365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.330564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.330954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.334656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.336890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.360777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 825/1667] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:15 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:15.362742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:15.380782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:15.384743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9b9568-ff0b-4a14-8e5b-9f170a9b1066 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:15.386770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:15.393160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17acc811-6b21-443b-8d67-27745341e057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:15.394471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acc811-6b21-443b-8d67-27745341e057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.394938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acc811-6b21-443b-8d67-27745341e057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.395402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17acc811-6b21-443b-8d67-27745341e057 admin admin] PATH_INFO: `/v3/projects/e7bdd2652a1e4a8eb093a9ee2adc31d1/users/fb54b7eb952043b7813a6ee04881da65/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.399154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acc811-6b21-443b-8d67-27745341e057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bdd2652a1e4a8eb093a9ee2adc31d1, user_id=fb54b7eb952043b7813a6ee04881da65, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.403635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17acc811-6b21-443b-8d67-27745341e057 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.416801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:15.425390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 843/1668] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:15 2022] PUT /identity/v3/projects/e7bdd2652a1e4a8eb093a9ee2adc31d1/users/fb54b7eb952043b7813a6ee04881da65/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:15.432495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.433418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.433791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.436814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb99ff-0726-44ab-a2fa-63448e163e74 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:15.452116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:15.501439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] MFA Rules not processed for user `fb54b7eb952043b7813a6ee04881da65`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:15.504246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:15.552861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:15.553736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.554014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.554756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] PATH_INFO: `/v3/users/aba55e3e1a4048abb617c3063f227dbe` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.558378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba55e3e1a4048abb617c3063f227dbe)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.563057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.658933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba55e3e1a4048abb617c3063f227dbe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:15.659740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba55e3e1a4048abb617c3063f227dbe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:15.660526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba55e3e1a4048abb617c3063f227dbe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:15.677031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba55e3e1a4048abb617c3063f227dbe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:15.678786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 844/1669] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:15.693512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba55e3e1a4048abb617c3063f227dbe'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:15.694573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:15.723150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 826/1670] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:15 2022] DELETE /identity/v3/users/aba55e3e1a4048abb617c3063f227dbe => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:15.770894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a274ba4f-dfaf-46ba-a001-f21110ae637a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:15.859237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:15.860106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:15.860628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:15.861819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:15.864463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:15.917473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:15.919900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:15.974152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:15.979112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:15.980686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:15.992543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 845/1671] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:16.024482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31826f2-8def-44c2-a7a4-ac7fb099e788 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.034040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.034703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.034992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.035271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] PATH_INFO: `/v3/projects/df0cedf742a54210a64ad18a30437959` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.036661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=df0cedf742a54210a64ad18a30437959)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.040278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.050464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.052398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.069708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:16.081517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2aee8-9b34-4da8-8cf9-dcaf681fe90b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.087665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.088316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.088616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.089033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] PATH_INFO: `/v3/users/d44ab34fe9824eb89f8cb5b68d38bd97` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.090494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d44ab34fe9824eb89f8cb5b68d38bd97)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.093036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.100418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df0cedf742a54210a64ad18a30437959', 'request_id': 'req-5d1477b7-03dc-4b69-b277-0756265dccbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.107934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df0cedf742a54210a64ad18a30437959', 'request_id': 'req-5d1477b7-03dc-4b69-b277-0756265dccbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.115151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df0cedf742a54210a64ad18a30437959', 'request_id': 'req-5d1477b7-03dc-4b69-b277-0756265dccbf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.128002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] The token cache is being invalidate because project df0cedf742a54210a64ad18a30437959 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:16.128316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.129085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.133479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 827/1672] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:15 2022] DELETE /identity/v3/projects/df0cedf742a54210a64ad18a30437959 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:16.134539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.135938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.144899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd44ab34fe9824eb89f8cb5b68d38bd97'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.152087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd44ab34fe9824eb89f8cb5b68d38bd97'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.153350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd44ab34fe9824eb89f8cb5b68d38bd97'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.160992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd44ab34fe9824eb89f8cb5b68d38bd97'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.161672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd44ab34fe9824eb89f8cb5b68d38bd97'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.180508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 846/1673] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:16 2022] DELETE /identity/v3/users/d44ab34fe9824eb89f8cb5b68d38bd97 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:16.181261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.182619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.209058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1477b7-03dc-4b69-b277-0756265dccbf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.253061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d528a-ce92-4865-83e4-ed652207d120 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.259279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.259927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.260206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.260479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] PATH_INFO: `/v3/users/a48fbc9cad4342f6bcc464758319b71e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.261686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48fbc9cad4342f6bcc464758319b71e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.264432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.294702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.295541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.295832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.296129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] PATH_INFO: `/v3/users/7b0e50b51d954c76955f5c8275a0242c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.297938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0e50b51d954c76955f5c8275a0242c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.299997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.334265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48fbc9cad4342f6bcc464758319b71e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.348351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48fbc9cad4342f6bcc464758319b71e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.349092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48fbc9cad4342f6bcc464758319b71e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.358864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48fbc9cad4342f6bcc464758319b71e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.359929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bc6075a0-633d-4917-b08c-5876fd07df77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48fbc9cad4342f6bcc464758319b71e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.365037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:16.375498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0e50b51d954c76955f5c8275a0242c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.375798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0e50b51d954c76955f5c8275a0242c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.376881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0e50b51d954c76955f5c8275a0242c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.384583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 847/1674] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:16 2022] DELETE /identity/v3/users/a48fbc9cad4342f6bcc464758319b71e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:16.385910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0e50b51d954c76955f5c8275a0242c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.389775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.390168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.390451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.391167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:16.407765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7482949-9ddf-4e7b-9f79-47a12b7ac0d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0e50b51d954c76955f5c8275a0242c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.426092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:16.430267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.444737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 828/1675] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:16 2022] DELETE /identity/v3/users/7b0e50b51d954c76955f5c8275a0242c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:16.450135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.450399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.450705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.474508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:16.475739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.532408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 848/1676] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:16.533698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.535223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.546977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 829/1677] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:16.546977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.546977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.620605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e08fb-e0d3-46c8-a78b-69d489635daf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.622987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e420c070-156e-4ae7-bdf8-7e365948efc5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.627595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.628296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.628687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.629023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.658434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.660006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.672216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 830/1678] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:16.673725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.685408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b967-5147-4b9d-8e1f-952b29b54579 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.690132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.690787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.691266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.691552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.705884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.707309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.721637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 831/1679] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:16.722559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:16.723886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.740620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.741289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.741607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.742302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.748361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49526d08-7a37-4a7c-ad9f-cf438ca98537 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:16.753342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:16.754488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.754770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.755044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] PATH_INFO: `/v3/projects/83fda30f213a4d80b5ae0ce7bf67417c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.755453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.756349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=83fda30f213a4d80b5ae0ce7bf67417c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.758357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.816930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83fda30f213a4d80b5ae0ce7bf67417c', 'request_id': 'req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.827130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83fda30f213a4d80b5ae0ce7bf67417c', 'request_id': 'req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.838685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:16.844322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83fda30f213a4d80b5ae0ce7bf67417c', 'request_id': 'req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.846783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:16.852308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:16.856944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16546fdf-6af5-4c73-91e0-86bfb4b1c0b2 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:16.865727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] The token cache is being invalidate because project 83fda30f213a4d80b5ae0ce7bf67417c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:16.866172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.867687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-504d7cd5-9cec-4ea1-a8fb-9f0b6ef0ccdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:16.870939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 832/1680] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:16 2022] DELETE /identity/v3/projects/83fda30f213a4d80b5ae0ce7bf67417c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:16.872983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 849/1681] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:16.876375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:16.876963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:16.877617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:16.902502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:16.904336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:16.996982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 833/1682] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:17.005551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.006053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.006158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.056809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:17.057847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.086735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 850/1683] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:17.091353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.109059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c6eb71-c8d2-4501-8ad5-52eda9cdfa9f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.115134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.116038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.116429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.117151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.118662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:17.119310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.122869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.145779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 834/1684] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:32:17.147231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.158146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c33a1-5472-40c6-baea-e5c0504b8221 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.165907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.167217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.170277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.170635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.171584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.188104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.237253 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 851/1685] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:17 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:17.237877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.252087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d0928b-3480-4f76-b14a-de15a1ec40aa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.370046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.370665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.371025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.371265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.371988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.374095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.385162 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.388729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.405029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c601a04-5a74-4114-a3ab-213a317ca24f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.410292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 835/1686] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:17 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:17.410292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.415460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.416554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.417039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.417620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] PATH_INFO: `/v3/projects/1f17e062659d48e184518156ba1ba766` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.419962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f17e062659d48e184518156ba1ba766)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.422886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.423928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cca83-1686-41d0-8027-5e4b98be321b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.430284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.430861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.431816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.432297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.433047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.436861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.450629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 836/1687] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:17.454983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.457985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.475619 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c229062-ac9b-4c86-9ee4-5095f6b9a6cf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.487515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f17e062659d48e184518156ba1ba766', 'request_id': 'req-582a8fe8-5639-4098-ac44-93c18c44b2e6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.495538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f17e062659d48e184518156ba1ba766', 'request_id': 'req-582a8fe8-5639-4098-ac44-93c18c44b2e6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.506443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f17e062659d48e184518156ba1ba766', 'request_id': 'req-582a8fe8-5639-4098-ac44-93c18c44b2e6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.523954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] The token cache is being invalidate because project 1f17e062659d48e184518156ba1ba766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:17.524277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.524983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.526922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 852/1688] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:17 2022] DELETE /identity/v3/projects/1f17e062659d48e184518156ba1ba766 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:17.528272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.531512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.539388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:17.548592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.549248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.549877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.550165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] PATH_INFO: `/v3/projects/3fd116eb68cd415d9b265fcc0600a0b0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.554515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd116eb68cd415d9b265fcc0600a0b0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.558548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.613019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd116eb68cd415d9b265fcc0600a0b0', 'request_id': 'req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.620956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd116eb68cd415d9b265fcc0600a0b0', 'request_id': 'req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.621272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a8fe8-5639-4098-ac44-93c18c44b2e6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.628711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd116eb68cd415d9b265fcc0600a0b0', 'request_id': 'req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.640181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] The token cache is being invalidate because project 3fd116eb68cd415d9b265fcc0600a0b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:17.640468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.641191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.645804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 837/1689] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:17 2022] DELETE /identity/v3/projects/3fd116eb68cd415d9b265fcc0600a0b0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:17.645804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.645804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.705801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:17.716474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.718175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.719056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.719520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.720251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.724504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.752869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 853/1690] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:17.753977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.756744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.768153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad8f14-f65c-45b0-9d0e-b3e9ea3654ca admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.771570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd39fc-22e6-4232-af3b-8a3e2fcd0320 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.773033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.773699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.773979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.774257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] PATH_INFO: `/v3/users/99073d9acc2542ba9a0c5e8f56af25ce` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.775453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99073d9acc2542ba9a0c5e8f56af25ce)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.776386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.777007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.777290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.777403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.779015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] PATH_INFO: `/v3/users/fb54b7eb952043b7813a6ee04881da65` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:17.779484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb54b7eb952043b7813a6ee04881da65)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:17.781477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:17.838288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99073d9acc2542ba9a0c5e8f56af25ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.838726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb54b7eb952043b7813a6ee04881da65'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.839024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99073d9acc2542ba9a0c5e8f56af25ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.840209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99073d9acc2542ba9a0c5e8f56af25ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.849174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb54b7eb952043b7813a6ee04881da65'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.849483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99073d9acc2542ba9a0c5e8f56af25ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.849860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb54b7eb952043b7813a6ee04881da65'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.858478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99073d9acc2542ba9a0c5e8f56af25ce'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.858654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb54b7eb952043b7813a6ee04881da65'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.859299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb54b7eb952043b7813a6ee04881da65'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:17.882694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 854/1691] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:17 2022] DELETE /identity/v3/users/fb54b7eb952043b7813a6ee04881da65 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:17.883799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.885835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.890825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 838/1692] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:17 2022] DELETE /identity/v3/users/99073d9acc2542ba9a0c5e8f56af25ce => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:17.893894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:17.893894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:17.904378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:17.957469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42649789-eb2f-4ff3-9074-bb0a07308017 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.980991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943b9e49-920c-4dad-b005-5c0cb69fe39b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:17.990080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99983c7-a11c-45a4-b4d6-7d4c0d3c3841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:17.991105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99983c7-a11c-45a4-b4d6-7d4c0d3c3841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:17.991576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99983c7-a11c-45a4-b4d6-7d4c0d3c3841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:17.992119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99983c7-a11c-45a4-b4d6-7d4c0d3c3841 admin admin] PATH_INFO: `/v3/projects/e9e82620b4be4832b6437b9e1955502a/users/8d9dc92d38d543eb8c46571169d6a1a7/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.014724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99983c7-a11c-45a4-b4d6-7d4c0d3c3841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9e82620b4be4832b6437b9e1955502a, user_id=8d9dc92d38d543eb8c46571169d6a1a7, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:18.019366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99983c7-a11c-45a4-b4d6-7d4c0d3c3841 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:18.025690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:18.027562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.028215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.029072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.032029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.037229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 839/1693] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:17 2022] PUT /identity/v3/projects/e9e82620b4be4832b6437b9e1955502a/users/8d9dc92d38d543eb8c46571169d6a1a7/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:18.052348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.052770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.053458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.132046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:18.140599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] MFA Rules not processed for user `8d9dc92d38d543eb8c46571169d6a1a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:18.142348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.184957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:18.186999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:18.200475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 855/1694] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:18.315723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 840/1695] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:18.320442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:18.322036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.340267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded1ca4-19a9-48c1-9c10-7c6b64290826 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:18.435481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:18.436901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.437247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.437558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.438207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:18.440321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:18.472679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 856/1696] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:18 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:18.473991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:18.485363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b07959-716b-4b2b-a7af-a5fe81f07ff0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:18.567905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:18.573559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.584726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:18.585391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.585670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.585940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.586583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:18.594956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:18.600203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff875417-1271-47dd-a005-c4f427fcdde2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:18.611197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:18.614210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.614824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.615446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] PATH_INFO: `/v3/projects/e7bdd2652a1e4a8eb093a9ee2adc31d1` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.617390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bdd2652a1e4a8eb093a9ee2adc31d1)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:18.621195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:18.637916 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 841/1697] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:18 2022] POST /identity/v3/users => generated 293 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:18.639221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:18.640825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.663777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb30793-e873-4f6a-b88d-897cc60a9a8b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:18.676030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:18.677851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.681395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.681790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] PATH_INFO: `/v3/projects/5c1bed66c14045218e8cbb3687387c9e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.683687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c1bed66c14045218e8cbb3687387c9e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:18.686904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:18.696191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:18.784317 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bdd2652a1e4a8eb093a9ee2adc31d1', 'request_id': 'req-151d14b6-099b-43e3-9703-d3dd6a0425fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.798604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bdd2652a1e4a8eb093a9ee2adc31d1', 'request_id': 'req-151d14b6-099b-43e3-9703-d3dd6a0425fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.802743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c1bed66c14045218e8cbb3687387c9e', 'request_id': 'req-a56d8f05-2565-41da-bbb3-b55db54ce5cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.812361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bdd2652a1e4a8eb093a9ee2adc31d1', 'request_id': 'req-151d14b6-099b-43e3-9703-d3dd6a0425fc'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.815117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c1bed66c14045218e8cbb3687387c9e', 'request_id': 'req-a56d8f05-2565-41da-bbb3-b55db54ce5cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.826603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] The token cache is being invalidate because project e7bdd2652a1e4a8eb093a9ee2adc31d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:18.827765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.831496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.832025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 857/1698] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:18 2022] DELETE /identity/v3/projects/e7bdd2652a1e4a8eb093a9ee2adc31d1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:18.832908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:18.834357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.850712 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c1bed66c14045218e8cbb3687387c9e', 'request_id': 'req-a56d8f05-2565-41da-bbb3-b55db54ce5cf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.877691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] The token cache is being invalidate because project 5c1bed66c14045218e8cbb3687387c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:18.877984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.880761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a56d8f05-2565-41da-bbb3-b55db54ce5cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:18.882530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 842/1699] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:18 2022] DELETE /identity/v3/projects/5c1bed66c14045218e8cbb3687387c9e => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:18.883844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b22bc-befb-4c48-89e0-4d017cfa3bdc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:18.884128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b22bc-befb-4c48-89e0-4d017cfa3bdc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:18.884398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744b22bc-befb-4c48-89e0-4d017cfa3bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:18.891798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:18.918024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-744b22bc-befb-4c48-89e0-4d017cfa3bdc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:18.920238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-744b22bc-befb-4c48-89e0-4d017cfa3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:18.925355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151d14b6-099b-43e3-9703-d3dd6a0425fc admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.003606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 843/1700] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:19.005408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b9f6b-b240-4b82-ac21-1021b30250f6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.005770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b9f6b-b240-4b82-ac21-1021b30250f6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.006056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b9f6b-b240-4b82-ac21-1021b30250f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.033405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8c2b9f6b-b240-4b82-ac21-1021b30250f6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:19.034601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8c2b9f6b-b240-4b82-ac21-1021b30250f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.046736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 844/1701] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:19.048198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.048485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.048865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.063820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67ee8f-b86e-4cae-9005-d36dea20c68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.064455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67ee8f-b86e-4cae-9005-d36dea20c68e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.064737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67ee8f-b86e-4cae-9005-d36dea20c68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.065013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67ee8f-b86e-4cae-9005-d36dea20c68e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.065700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67ee8f-b86e-4cae-9005-d36dea20c68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.067604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67ee8f-b86e-4cae-9005-d36dea20c68e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.076912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:19.078204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.082358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 858/1702] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:19.083712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.083997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.084273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.096019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 845/1703] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:19.096019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.096656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.105045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:19.106845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.114088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8804a39-5e9e-4975-b03f-735d4ec074e6 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.119560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.120162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.120786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.120786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] PATH_INFO: `/v3/projects/3909261445ea436cbf82fbd07d3e1609/users/3e2f554159e34971849f1fda15b2df5d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.122320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 859/1704] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:19.124129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.141012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3909261445ea436cbf82fbd07d3e1609, user_id=3e2f554159e34971849f1fda15b2df5d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.142202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0532d-c5d2-4d60-bd50-454ebb92ef55 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.144063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.147971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.148626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.148902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.149166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.149942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:19.150338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.152051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.168113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 846/1705] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:19 2022] PUT /identity/v3/projects/3909261445ea436cbf82fbd07d3e1609/users/3e2f554159e34971849f1fda15b2df5d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:19.169495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 860/1706] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:19.170473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.170586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.194814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df1d9ab-1da0-4728-985e-fa29b05e1671 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.252879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:19.259038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa3eb1-dd66-41f1-a825-de78578ae8f7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.266068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.266612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.266920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.267332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.268179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:19.268486 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.270130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.276340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.276953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.277230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.277538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.278305 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.280050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.283976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 847/1707] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:19.286263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.299473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 861/1708] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:19.299473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.299740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d603ca4-2ab9-4330-9997-771f3afb108a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.303982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.304574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.304856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.305132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.305808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.307449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c077e7b2-79d9-4935-9f84-c9df48534734 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.307915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.313105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.313728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.314009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.314311 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.314939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.316824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.337386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 848/1709] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:19.338037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.351589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 862/1710] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:19.351589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.406756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:19.424725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01ad60-5c56-45f0-ac97-77b86c0c09bd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.430046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.430711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.431527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.431815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] PATH_INFO: `/v3/projects/3909261445ea436cbf82fbd07d3e1609/users/3e2f554159e34971849f1fda15b2df5d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.434821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3909261445ea436cbf82fbd07d3e1609, user_id=3e2f554159e34971849f1fda15b2df5d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.436153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a38314-7742-4991-87ba-d7421aea2717 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.439203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.443266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bb39a9-becf-4db7-a67e-bafede1d463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.444512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb39a9-becf-4db7-a67e-bafede1d463f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.445060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb39a9-becf-4db7-a67e-bafede1d463f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.445526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bb39a9-becf-4db7-a67e-bafede1d463f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.446179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb39a9-becf-4db7-a67e-bafede1d463f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.448185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bb39a9-becf-4db7-a67e-bafede1d463f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.455547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 849/1711] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:19 2022] PUT /identity/v3/projects/3909261445ea436cbf82fbd07d3e1609/users/3e2f554159e34971849f1fda15b2df5d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:19.457278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.481813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 863/1712] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:19.483866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.484156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.484939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.527319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] MFA Rules not processed for user `3e2f554159e34971849f1fda15b2df5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:19.529457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.572087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac164e5-74cb-4d61-a608-3a5f6c450bf4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.581027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.583303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.584096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.586167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.588186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.592460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.624496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 850/1713] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:19.625520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.628432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 864/1714] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:19.629765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.637851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f2231-704b-4560-a558-3735d2631087 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.644007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b28a5-4a9f-49fc-aeca-61d64954959b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.645804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.646663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.647051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.647568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.648625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.650755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.653728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.654706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.655166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.655819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.656693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.659049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.665524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 851/1715] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:19.666520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.669263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.676377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 865/1716] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:19.677788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.692213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65332222-09e3-4055-9936-58c786bcacd1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.697132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.697771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.698054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.698327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.699103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.700928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.719045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:19.724936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 866/1717] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:19.725942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.740606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358eecc-49fd-47bf-bcd0-98260287551a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.745897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.747020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.747328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.747579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.748240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.749907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.755934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037cca51-d5b1-4c36-83f3-765fd2c51ae2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.763657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 867/1718] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:19.764750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.767152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.767798 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.768095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.768356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.774224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:19.784385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebabf6-3a5b-40b0-b856-18ee07721b79 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:19.807820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:19.808504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.808777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.809888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] PATH_INFO: `/v3/projects/bea6e269a36046de8e301035768603fb/users/71635a8c51db4010ab2dc38a2114f147/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.840489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea6e269a36046de8e301035768603fb, user_id=71635a8c51db4010ab2dc38a2114f147, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.845328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.878542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 868/1719] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:19 2022] PUT /identity/v3/projects/bea6e269a36046de8e301035768603fb/users/71635a8c51db4010ab2dc38a2114f147/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:19.889959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:19.915021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:19.920353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61b629-7442-4412-9b61-e116fe811d15 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:19.944473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 852/1720] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:19.946454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:19.946740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:19.947043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:19.999795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] MFA Rules not processed for user `71635a8c51db4010ab2dc38a2114f147`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:20.001474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.003068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:20.013419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1433ad-dac9-4b76-882e-81f67263034e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.020642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a24db-13de-4885-89f4-2c3c1558ad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.021478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a24db-13de-4885-89f4-2c3c1558ad80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.021766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a24db-13de-4885-89f4-2c3c1558ad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.022041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a24db-13de-4885-89f4-2c3c1558ad80 admin admin] PATH_INFO: `/v3/projects/d5a6e4eaadf74e40a9a9186ccc65f61c/users/b95742174c264ffe921eb2c4f5ac5318/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.042892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a24db-13de-4885-89f4-2c3c1558ad80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5a6e4eaadf74e40a9a9186ccc65f61c, user_id=b95742174c264ffe921eb2c4f5ac5318, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.048269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a24db-13de-4885-89f4-2c3c1558ad80 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.092562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 869/1721] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:19 2022] PUT /identity/v3/projects/d5a6e4eaadf74e40a9a9186ccc65f61c/users/b95742174c264ffe921eb2c4f5ac5318/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:20.102896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.103184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.103458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.116360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 853/1722] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:20.120432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.134729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61339554-1a80-4e28-ab64-40954805fbe9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.142319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] MFA Rules not processed for user `b95742174c264ffe921eb2c4f5ac5318`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:20.144199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.211949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.212693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.213035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.213768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.214171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.216899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.226479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 870/1723] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:20.230467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.243924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800a653-4e19-4c5d-8678-55abdc195d15 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.250049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.250676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.250958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.251230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.251868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.253951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.279265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 854/1724] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/projects => generated 383 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:20.284229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.304582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:20.327161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 871/1725] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:20.334313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.421850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c76e6e4-24f9-4a9a-b465-383727bce912 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.446187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43b853-6350-44cb-8fac-c11a6f3d12e9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.467547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.468177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.468459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.468731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.469440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.471509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.497904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 872/1726] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/users => generated 298 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:20.501741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.506225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.506912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.507277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.507619 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.508265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.510401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.511738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8754a-896e-4295-aee1-39f61501064b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.517070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.517723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.518023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.518310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.518995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.520651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.531372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 873/1727] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:20.532304 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 855/1728] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/users => generated 298 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:20.535447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.536099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.548170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d418d9-67b3-454f-9daa-d5186605f968 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.548360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf846160-be8a-4f30-a667-9d282e137b83 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.550812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.551478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.551761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.551869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.552035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] PATH_INFO: `/v3/projects/033c56742b12413ebd214130df8a3266/users/45513bf5b8f24247ad0944988088ad81/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.552472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.552799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.553026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.553707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.556490 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.569415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 856/1729] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:20.569642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=033c56742b12413ebd214130df8a3266, user_id=45513bf5b8f24247ad0944988088ad81, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.573057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.573288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.583863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:20.584805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da38cf-3eeb-4b9e-a31b-198326b7cafb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.589344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 874/1730] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:20 2022] PUT /identity/v3/projects/033c56742b12413ebd214130df8a3266/users/45513bf5b8f24247ad0944988088ad81/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:20.593351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.654218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.654892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.655176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.655452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] PATH_INFO: `/v3/projects/cc312f1b8a294c55ba4c507269e869b8/users/621fdf723ec34b4aa9165505b954e995/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.672028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc312f1b8a294c55ba4c507269e869b8, user_id=621fdf723ec34b4aa9165505b954e995, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.673673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bb1ca0-ce05-4c47-9dfe-8948b497d425 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.674758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.680496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.681161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.681766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.682169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.682942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.685763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.689788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 857/1731] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:20 2022] PUT /identity/v3/projects/cc312f1b8a294c55ba4c507269e869b8/users/621fdf723ec34b4aa9165505b954e995/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:20.693392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.700225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 875/1732] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:20.701174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.712825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68126be3-b45a-4fcf-884f-92a2876f9313 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.716027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:20.773791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae745-994f-4ee1-bf32-552671436351 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.779704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.780396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.780738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.780880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630e6f5-8436-4853-866e-34ed8f36695c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.781124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.781493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630e6f5-8436-4853-866e-34ed8f36695c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.781776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630e6f5-8436-4853-866e-34ed8f36695c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.781984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.782107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630e6f5-8436-4853-866e-34ed8f36695c admin admin] PATH_INFO: `/v3/projects/033c56742b12413ebd214130df8a3266/users/45513bf5b8f24247ad0944988088ad81/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.783697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.784540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630e6f5-8436-4853-866e-34ed8f36695c admin admin] RBAC: Authorizing `identity:create_grant(project_id=033c56742b12413ebd214130df8a3266, user_id=45513bf5b8f24247ad0944988088ad81, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.787806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630e6f5-8436-4853-866e-34ed8f36695c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.795393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 858/1733] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:20.799369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.799611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 876/1734] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:20 2022] PUT /identity/v3/projects/033c56742b12413ebd214130df8a3266/users/45513bf5b8f24247ad0944988088ad81/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:20.810872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c61b8c-cfcf-40d8-add8-2aaad8586b33 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:20.811540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.811748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.812023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.845104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] MFA Rules not processed for user `45513bf5b8f24247ad0944988088ad81`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:20.846367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.879909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a8742-5ca7-49bc-bc70-252d6b546ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:20.880794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a8742-5ca7-49bc-bc70-252d6b546ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.880976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a8742-5ca7-49bc-bc70-252d6b546ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.881280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a8742-5ca7-49bc-bc70-252d6b546ce8 admin admin] PATH_INFO: `/v3/projects/cc312f1b8a294c55ba4c507269e869b8/users/621fdf723ec34b4aa9165505b954e995/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.883366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a8742-5ca7-49bc-bc70-252d6b546ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc312f1b8a294c55ba4c507269e869b8, user_id=621fdf723ec34b4aa9165505b954e995, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:20.887766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a8742-5ca7-49bc-bc70-252d6b546ce8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:20.904714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 859/1735] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:20 2022] PUT /identity/v3/projects/cc312f1b8a294c55ba4c507269e869b8/users/621fdf723ec34b4aa9165505b954e995/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:20.914937 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:20.915223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:20.915773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:20.933351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 877/1736] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:20.945447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:20.951438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] MFA Rules not processed for user `621fdf723ec34b4aa9165505b954e995`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:20.952889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:20.967025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:21.030602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef83bf28-3fda-4998-9b3b-f2a5eb618f9a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:21.047044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 860/1737] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:20 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:21.070824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:21.071931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:21.107502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:21.108202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:21.108777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:21.109082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:21.119556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:21.149031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa87675-567c-42f4-bc4f-c120b611febc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:21.153468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:21.154781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:21.154781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:21.154781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:21.165342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:21.166935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:21.180532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:21.182149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:21.191585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 861/1738] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:32:21.194201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 878/1739] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:32:21.564809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:21.575980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a84124-f728-472c-b5ed-5a9194e7dcda service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:21.583563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb236f5-71f8-4728-bd3e-cec3117f8d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:21.584336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb236f5-71f8-4728-bd3e-cec3117f8d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:21.585403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb236f5-71f8-4728-bd3e-cec3117f8d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:21.585905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb236f5-71f8-4728-bd3e-cec3117f8d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:21.661246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb236f5-71f8-4728-bd3e-cec3117f8d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:21.664608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb236f5-71f8-4728-bd3e-cec3117f8d8e service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:21.676516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 862/1740] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:22.815381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:22.816828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:22.921709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c813f2-546f-4cdc-80e7-b56a9ab700cc service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:22.927879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:22.928536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:22.928818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:22.929095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] PATH_INFO: `/v3/users/d7c71fee306a494c9caba3dc0869e816` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:22.930293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c71fee306a494c9caba3dc0869e816)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:22.932279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:22.977949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:23.014865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c71fee306a494c9caba3dc0869e816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:23.023580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c71fee306a494c9caba3dc0869e816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:23.024557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c71fee306a494c9caba3dc0869e816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:23.034136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c71fee306a494c9caba3dc0869e816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:23.034848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c71fee306a494c9caba3dc0869e816'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:23.062049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 879/1741] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:22 2022] DELETE /identity/v3/users/d7c71fee306a494c9caba3dc0869e816 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:23.064331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:23.064331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:23.064331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:23.085131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:23.086473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:23.162562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 863/1742] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:23.177756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:23.177756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:23.242727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f9990-cdb7-4ff2-81c1-682fc9f2c484 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:23.247628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:23.248540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:23.248827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:23.249180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:23.272559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:23.274653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:23.288425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 880/1743] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:24.044636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:24.046421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:24.060568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c0e8d-6fb9-4a0c-86ae-5bfb06b3e447 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:24.067463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:24.069501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:24.069938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:24.070363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] PATH_INFO: `/v3/users/8d9dc92d38d543eb8c46571169d6a1a7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:24.071920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9dc92d38d543eb8c46571169d6a1a7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:24.074072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:24.146767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9dc92d38d543eb8c46571169d6a1a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.147723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9dc92d38d543eb8c46571169d6a1a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.148763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9dc92d38d543eb8c46571169d6a1a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.159784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9dc92d38d543eb8c46571169d6a1a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.169274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-92965182-be65-4d6d-98e0-37351fcb226a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9dc92d38d543eb8c46571169d6a1a7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.175842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:24.177742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:24.182438 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:24.208145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cfd3f3-e854-4564-bc14-ba44b0d1d791 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:24.215650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:24.216307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:24.216584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:24.216856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] PATH_INFO: `/v3/projects/5ff0ab9f8f224120867b32a3e5596562` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:24.218194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff0ab9f8f224120867b32a3e5596562)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:24.220268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:24.224504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 864/1744] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:24 2022] DELETE /identity/v3/users/8d9dc92d38d543eb8c46571169d6a1a7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:24.229001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:24.229561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:24.231447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:24.253626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:24.266365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:24.300154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff0ab9f8f224120867b32a3e5596562', 'request_id': 'req-e2f7a51f-169b-453a-ad49-f611711adc26'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.306936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff0ab9f8f224120867b32a3e5596562', 'request_id': 'req-e2f7a51f-169b-453a-ad49-f611711adc26'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.315819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff0ab9f8f224120867b32a3e5596562', 'request_id': 'req-e2f7a51f-169b-453a-ad49-f611711adc26'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.329482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] The token cache is being invalidate because project 5ff0ab9f8f224120867b32a3e5596562 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:24.329783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.330441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.332237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 881/1745] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:24 2022] DELETE /identity/v3/projects/5ff0ab9f8f224120867b32a3e5596562 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:24.337105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 865/1746] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:24.338999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:24.341846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:24.355007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:24.357409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:24.408350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:24.469088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7a51f-169b-453a-ad49-f611711adc26 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:24.474665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:24.475420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:24.475794 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:24.476168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] PATH_INFO: `/v3/users/a027f75789b841989272652b315e5d14` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:24.477521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a027f75789b841989272652b315e5d14)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:24.479752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:24.487835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5643d5-1b04-4532-8c78-cecff8aa198c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:24.547969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a027f75789b841989272652b315e5d14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.560373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a027f75789b841989272652b315e5d14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.561384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a027f75789b841989272652b315e5d14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.571227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a027f75789b841989272652b315e5d14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.572146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a027f75789b841989272652b315e5d14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:24.587802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:24.588427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:24.588705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:24.589252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:24.593233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:24.625557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:24.627010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:24.645635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 882/1747] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:24 2022] DELETE /identity/v3/users/a027f75789b841989272652b315e5d14 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:24.653092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 866/1748] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:25.383907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:25.386448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:25.492106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:25.494352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:25.512286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12102d38-ec27-402f-ba30-ba6b98826081 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:25.544385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee630a2-a9e6-4aee-b4b5-7ac70ff5f784 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:25.550671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:25.551646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:25.551774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:25.552135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] PATH_INFO: `/v3/projects/e9e82620b4be4832b6437b9e1955502a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:25.553787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9e82620b4be4832b6437b9e1955502a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:25.556430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:25.577585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:25.632396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9e82620b4be4832b6437b9e1955502a', 'request_id': 'req-608cc31b-0f99-47f8-9530-b2923e7cf7b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.634748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:25.638914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:25.639642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:25.639921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:25.640185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] PATH_INFO: `/v3/projects/9bbdc6b8bc1f4201bbf43f0a8b47fca7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:25.642380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bbdc6b8bc1f4201bbf43f0a8b47fca7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:25.643749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9e82620b4be4832b6437b9e1955502a', 'request_id': 'req-608cc31b-0f99-47f8-9530-b2923e7cf7b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.647897 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:25.659847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9e82620b4be4832b6437b9e1955502a', 'request_id': 'req-608cc31b-0f99-47f8-9530-b2923e7cf7b8'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.680883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] The token cache is being invalidate because project e9e82620b4be4832b6437b9e1955502a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:25.681183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.681942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.687366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 883/1749] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:25 2022] DELETE /identity/v3/projects/e9e82620b4be4832b6437b9e1955502a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:25.688195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:25.697662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:25.757543 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bbdc6b8bc1f4201bbf43f0a8b47fca7', 'request_id': 'req-0018395c-2789-4ab0-97ed-ad67a9a16a01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.775353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bbdc6b8bc1f4201bbf43f0a8b47fca7', 'request_id': 'req-0018395c-2789-4ab0-97ed-ad67a9a16a01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.784576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bbdc6b8bc1f4201bbf43f0a8b47fca7', 'request_id': 'req-0018395c-2789-4ab0-97ed-ad67a9a16a01'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.810059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] The token cache is being invalidate because project 9bbdc6b8bc1f4201bbf43f0a8b47fca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:25.810361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.812257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:25.814880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 867/1750] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:25 2022] DELETE /identity/v3/projects/9bbdc6b8bc1f4201bbf43f0a8b47fca7 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:25.820295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:25.822750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cc31b-0f99-47f8-9530-b2923e7cf7b8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:25.829333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:25.948662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018395c-2789-4ab0-97ed-ad67a9a16a01 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:25.958703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:25.959656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:25.960059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:25.960359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] PATH_INFO: `/v3/users/cc4a9a08e0684056bb3bb21a08b79801` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:25.961594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc4a9a08e0684056bb3bb21a08b79801)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:25.966233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:25.967010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:25.967349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:25.967688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] PATH_INFO: `/v3/users/3e2f554159e34971849f1fda15b2df5d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:25.969098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2f554159e34971849f1fda15b2df5d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:25.970160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:25.976562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:26.033668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc4a9a08e0684056bb3bb21a08b79801'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.034327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc4a9a08e0684056bb3bb21a08b79801'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.035019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc4a9a08e0684056bb3bb21a08b79801'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.046739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc4a9a08e0684056bb3bb21a08b79801'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.069156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc4a9a08e0684056bb3bb21a08b79801'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.087339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:26.101038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2f554159e34971849f1fda15b2df5d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.102198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 868/1751] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:25 2022] DELETE /identity/v3/users/cc4a9a08e0684056bb3bb21a08b79801 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:26.103156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:26.104626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.134295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2f554159e34971849f1fda15b2df5d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.136074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2f554159e34971849f1fda15b2df5d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.169192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2f554159e34971849f1fda15b2df5d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.170853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6dae1164-e04d-4bcb-ab7d-7f7dd8504c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2f554159e34971849f1fda15b2df5d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.216226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1d4b76-b5a1-4ff1-89be-5366799aa163 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:26.222734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 884/1752] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:25 2022] DELETE /identity/v3/users/3e2f554159e34971849f1fda15b2df5d => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:26.325751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:26.326903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.327331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.327757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] PATH_INFO: `/v3/users/71635a8c51db4010ab2dc38a2114f147` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.329260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71635a8c51db4010ab2dc38a2114f147)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:26.332326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:26.457039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71635a8c51db4010ab2dc38a2114f147'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.458648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71635a8c51db4010ab2dc38a2114f147'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.459838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71635a8c51db4010ab2dc38a2114f147'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.472783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71635a8c51db4010ab2dc38a2114f147'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.488457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71635a8c51db4010ab2dc38a2114f147'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.532613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 869/1753] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:26 2022] DELETE /identity/v3/users/71635a8c51db4010ab2dc38a2114f147 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:26.534419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.534795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.534971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.553509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:26.566535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:26.571867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.656007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 885/1754] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:26.673552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:26.675730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.695520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:26.701395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.719167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584fc6c2-46c3-45c7-a7e5-5a3469217b68 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:26.722632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:26.728427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:26.729388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.730224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.730879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] PATH_INFO: `/v3/projects/ec2dba916db34c3b987bfdda1bebf80e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.733690 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2dba916db34c3b987bfdda1bebf80e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:26.736502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:26.770057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4e3a5-9bff-4cd1-8c47-c5fb1bc6ba8b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:26.776383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58e284-e204-4fe5-a74f-b1db410149db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:26.778437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58e284-e204-4fe5-a74f-b1db410149db service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.793622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58e284-e204-4fe5-a74f-b1db410149db service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.796247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58e284-e204-4fe5-a74f-b1db410149db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.804214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2dba916db34c3b987bfdda1bebf80e', 'request_id': 'req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.813226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2dba916db34c3b987bfdda1bebf80e', 'request_id': 'req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.815398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58e284-e204-4fe5-a74f-b1db410149db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:26.818392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58e284-e204-4fe5-a74f-b1db410149db service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:26.822774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2dba916db34c3b987bfdda1bebf80e', 'request_id': 'req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.833011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 870/1755] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:26.842751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] The token cache is being invalidate because project ec2dba916db34c3b987bfdda1bebf80e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:26.843130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.843850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:26.845456 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 886/1756] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:26 2022] DELETE /identity/v3/projects/ec2dba916db34c3b987bfdda1bebf80e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:26.850959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a28c5-6cbd-475b-aef1-3f3ab6f65086 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.851243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a28c5-6cbd-475b-aef1-3f3ab6f65086 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.851533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a28c5-6cbd-475b-aef1-3f3ab6f65086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:26.880605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e75a28c5-6cbd-475b-aef1-3f3ab6f65086 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:26.882302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e75a28c5-6cbd-475b-aef1-3f3ab6f65086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.978513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:26.980535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 871/1757] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:26.981392 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:26.982013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49561-f289-4483-9262-134a5191c792 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:26.982301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49561-f289-4483-9262-134a5191c792 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:26.982574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc49561-f289-4483-9262-134a5191c792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.010064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be4f57a-c5f4-4f94-82ac-18afbef5dc14 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.017823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.018969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.020113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.020366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6fc49561-f289-4483-9262-134a5191c792 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:27.020786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] PATH_INFO: `/v3/projects/3909261445ea436cbf82fbd07d3e1609` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.022939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=3909261445ea436cbf82fbd07d3e1609)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.023826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6fc49561-f289-4483-9262-134a5191c792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.024965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.040982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 872/1758] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:27.047161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc49561-f289-4483-9262-134a5191c792 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.060969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc49561-f289-4483-9262-134a5191c792 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.067279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.068239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.068533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.068841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.069518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:27.069872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.072745 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.093482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 873/1759] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:27.094777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.096507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3909261445ea436cbf82fbd07d3e1609', 'request_id': 'req-3f11d0d8-743c-4a71-9d9c-377cb43462ee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.107963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3909261445ea436cbf82fbd07d3e1609', 'request_id': 'req-3f11d0d8-743c-4a71-9d9c-377cb43462ee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.108328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43a5bb-3811-4e08-9b7c-ad76b65929c9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.120448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3909261445ea436cbf82fbd07d3e1609', 'request_id': 'req-3f11d0d8-743c-4a71-9d9c-377cb43462ee'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.143780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] The token cache is being invalidate because project 3909261445ea436cbf82fbd07d3e1609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:27.144420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.145600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-3f11d0d8-743c-4a71-9d9c-377cb43462ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.148111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 887/1760] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:26 2022] DELETE /identity/v3/projects/3909261445ea436cbf82fbd07d3e1609 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:27.153610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfc850-9770-4ca8-974f-7915901a2c41 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.154306 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfc850-9770-4ca8-974f-7915901a2c41 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.154582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dfc850-9770-4ca8-974f-7915901a2c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.189302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.190055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.190336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.190611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.191251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.193366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.197422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-30dfc850-9770-4ca8-974f-7915901a2c41 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:27.203861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-30dfc850-9770-4ca8-974f-7915901a2c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.226563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 888/1761] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:27.228566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.229028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.229699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.239155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 874/1762] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:27 2022] POST /identity/v3/projects => generated 363 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:27.243780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.247785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.290317 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:27.301549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:27.307264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.332018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:27.374531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd62af-305d-455c-aaa6-95a25c258d14 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.385547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.386183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.386464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.386739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.388027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.391488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.394189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 889/1763] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:27.395184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.396538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.410331 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67b8be-c36c-4cb8-be5c-0e8efdaf3fc6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.416815 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.417508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.417787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.418672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] PATH_INFO: `/v3/projects/bea6e269a36046de8e301035768603fb` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.419822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea6e269a36046de8e301035768603fb)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.421183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 875/1764] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:27 2022] POST /identity/v3/users => generated 282 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:27.421961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.423595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.425156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.477467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea6e269a36046de8e301035768603fb', 'request_id': 'req-4520fe8b-c40b-4e24-a9c0-78f8f2648895'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.490830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea6e269a36046de8e301035768603fb', 'request_id': 'req-4520fe8b-c40b-4e24-a9c0-78f8f2648895'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.492659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947de93d-982e-48f8-a2bf-9a645b8feb1d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.502869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea6e269a36046de8e301035768603fb', 'request_id': 'req-4520fe8b-c40b-4e24-a9c0-78f8f2648895'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.521640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] The token cache is being invalidate because project bea6e269a36046de8e301035768603fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:27.521930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.522607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.525300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 890/1765] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:27 2022] DELETE /identity/v3/projects/bea6e269a36046de8e301035768603fb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:27.526131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.527607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.562791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.563579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.563955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.564330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.567456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.611887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4520fe8b-c40b-4e24-a9c0-78f8f2648895 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.616987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.617911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.618523 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.618895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.619791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:27.620292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.623076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.639458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 891/1766] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:27.641755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.643639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.643960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.645119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.657863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 876/1767] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:27 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:27.658330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35959941-4e0c-4e2d-9098-bd9434401b0f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.658868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.660265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.663312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.664026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.664372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.664744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.665430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.667717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.676095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9526be-791b-4170-a209-18e3c2c0873e service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.682297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.683017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.683558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.683685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 892/1768] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:27.684418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] PATH_INFO: `/v3/users/621fdf723ec34b4aa9165505b954e995` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.685273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.685998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=621fdf723ec34b4aa9165505b954e995)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.688022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.699646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3851e-8cd0-46bc-abbb-384b8e7322ee admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:27.718247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:27.719053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:27.719345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:27.719619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:27.720397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:27.722774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:27.734163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:27.738818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:27.781404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 893/1769] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:27 2022] POST /identity/v3/projects => generated 375 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:27.788666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.805590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '621fdf723ec34b4aa9165505b954e995'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.807174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '621fdf723ec34b4aa9165505b954e995'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.808248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '621fdf723ec34b4aa9165505b954e995'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.825478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '621fdf723ec34b4aa9165505b954e995'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.843219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '621fdf723ec34b4aa9165505b954e995'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:27.881008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 877/1770] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:27 2022] DELETE /identity/v3/users/621fdf723ec34b4aa9165505b954e995 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:27.881677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:27.883714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:27.928963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf256905-50e0-48e2-b3cb-55e9cc686f69 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.009241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13905e0-1cd2-4d29-a857-c76efd945feb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.022104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.022748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.023033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.023305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.023947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.026027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.057421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.059915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.060423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.060872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.061431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.063454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.081180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 878/1771] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:27 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:28.082068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 894/1772] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:27 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:28.082649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.083572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.085602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.096010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438b824f-85f2-4b42-81bd-b5fa1536d707 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.102404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c15af-2454-4856-bbde-f09fd70c7cc3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.103512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.104328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.104707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.105219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.106725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.120348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.126733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 879/1773] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:28.128166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.129138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c593a-dfd5-401b-888e-428d290d7d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.131087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c593a-dfd5-401b-888e-428d290d7d93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.131558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c593a-dfd5-401b-888e-428d290d7d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.132032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c593a-dfd5-401b-888e-428d290d7d93 admin admin] PATH_INFO: `/v3/projects/b99c469c3ab149819915024f478d80a1/users/63b310e542594e3498bc5a0198d53b9f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.155836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c593a-dfd5-401b-888e-428d290d7d93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b99c469c3ab149819915024f478d80a1, user_id=63b310e542594e3498bc5a0198d53b9f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.159352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c593a-dfd5-401b-888e-428d290d7d93 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.165024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf583d-eaaf-49c8-ae51-17d1bf21d4e3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.178461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.179460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.179901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.180318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.181930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.187399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.196773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 895/1774] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:28 2022] PUT /identity/v3/projects/b99c469c3ab149819915024f478d80a1/users/63b310e542594e3498bc5a0198d53b9f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:28.207611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.207845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.208171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.221926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 880/1775] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:28.229394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.248317 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c88ab5-0b89-4822-b0f5-2d0d6ba8656c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.298407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] MFA Rules not processed for user `63b310e542594e3498bc5a0198d53b9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:28.299746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.366913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.367865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.368656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.369114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] PATH_INFO: `/v3/projects/5b5f57789fe74598914fe21b21991275/users/c9c07ccf684743a38f312ec5f465cd85/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.390172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:28.403630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:28.411195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b5f57789fe74598914fe21b21991275, user_id=c9c07ccf684743a38f312ec5f465cd85, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.417235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.418660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 896/1776] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:28.430688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.433623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.454273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 881/1777] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:28 2022] PUT /identity/v3/projects/5b5f57789fe74598914fe21b21991275/users/c9c07ccf684743a38f312ec5f465cd85/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:28.471792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.475346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.526173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b8032-1027-449a-a7f1-e53ef547c9cd None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.571068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be910a71-af82-4ba4-bf87-6c37b231ee2e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:28.577644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.578564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.579049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.579392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] PATH_INFO: `/v3/projects/cc312f1b8a294c55ba4c507269e869b8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.581529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc312f1b8a294c55ba4c507269e869b8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.583560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.641615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef25f9-3ef0-4f17-ba84-9e209df7d2d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:28.642574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef25f9-3ef0-4f17-ba84-9e209df7d2d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.642856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef25f9-3ef0-4f17-ba84-9e209df7d2d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.643123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef25f9-3ef0-4f17-ba84-9e209df7d2d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.681051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc312f1b8a294c55ba4c507269e869b8', 'request_id': 'req-b4cf90c7-12c4-45b0-bf4b-408c07f92681'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:28.693675 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc312f1b8a294c55ba4c507269e869b8', 'request_id': 'req-b4cf90c7-12c4-45b0-bf4b-408c07f92681'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:28.705001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc312f1b8a294c55ba4c507269e869b8', 'request_id': 'req-b4cf90c7-12c4-45b0-bf4b-408c07f92681'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:28.721519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] The token cache is being invalidate because project cc312f1b8a294c55ba4c507269e869b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:28.721802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:28.722693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b4cf90c7-12c4-45b0-bf4b-408c07f92681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:28.729371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 882/1778] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:28 2022] DELETE /identity/v3/projects/cc312f1b8a294c55ba4c507269e869b8 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:28.729371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.729371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.729371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.779788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef25f9-3ef0-4f17-ba84-9e209df7d2d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:28.782043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef25f9-3ef0-4f17-ba84-9e209df7d2d2 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:28.786480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] MFA Rules not processed for user `c9c07ccf684743a38f312ec5f465cd85`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:28.790027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.803410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 897/1779] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:28.805770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f1d2-cd7d-40ee-8684-4e92bd0cc168 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.806161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f1d2-cd7d-40ee-8684-4e92bd0cc168 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.806905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d848f1d2-cd7d-40ee-8684-4e92bd0cc168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:28.844931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d848f1d2-cd7d-40ee-8684-4e92bd0cc168 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:28.846202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d848f1d2-cd7d-40ee-8684-4e92bd0cc168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.906363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 883/1780] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:28.919350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:28.921848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:28.937177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:28.976860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 898/1781] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:28.987106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:28.988825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:28.989116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.040320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:29.042600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:29.060661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 899/1782] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:29.061920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.068691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:29.074961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf37d48-1a9b-487a-ad73-452513a7289b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.081612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.083212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.083497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.083769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.084420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:29.084737 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.086136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9564e1-e49a-4aa8-a650-2a5d3dd3e339 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.086434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.093280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.094346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.094702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.096920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.104576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 900/1783] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:29.109716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.113824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.116012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.127300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2734afd-546c-4f90-88b5-57e380f93dab admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.130131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 884/1784] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:29.149808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.151782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.152195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.153908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.154663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.160293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.240406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 901/1785] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:29 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:29.244955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.261348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e088a-fe09-4100-b139-0ccaa4b39217 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.374905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.375722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.376078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.376450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.377124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.384390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.428669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 885/1786] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:29 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:29.432810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.446720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a92f00-9646-4972-a886-a99757637454 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.475197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.475872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.476134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.476511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.477238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.479202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.491470 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 902/1787] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:29.497580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.511776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6f8622-8a30-4057-a8f5-9e915135bf67 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.523201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce2a16e-fe4e-4b5d-9085-ff079ed93942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.524111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2a16e-fe4e-4b5d-9085-ff079ed93942 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.524585 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2a16e-fe4e-4b5d-9085-ff079ed93942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.525033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce2a16e-fe4e-4b5d-9085-ff079ed93942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.526354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2a16e-fe4e-4b5d-9085-ff079ed93942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.529202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce2a16e-fe4e-4b5d-9085-ff079ed93942 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.541886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 886/1788] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:29.547716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.560563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9228a69-218b-4690-9709-3599cd0471f4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.565531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.566329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.566635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.566914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] PATH_INFO: `/v3/projects/99bf0889389d4d8f889b61f8ded5471b/users/9bf228a4b9144370baf1049347f3a0a1/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.601167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bf0889389d4d8f889b61f8ded5471b, user_id=9bf228a4b9144370baf1049347f3a0a1, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.604069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.639711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 903/1789] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:29 2022] PUT /identity/v3/projects/99bf0889389d4d8f889b61f8ded5471b/users/9bf228a4b9144370baf1049347f3a0a1/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:29.649958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.650398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.650532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.696990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] MFA Rules not processed for user `9bf228a4b9144370baf1049347f3a0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:29.698471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:29.781589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 887/1790] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:29.794229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.859507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c5ad4-eee7-475d-a3e9-eec7d01ba70b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.863154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56cd3ce-4cb2-49c0-9314-7e46433ee725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.863743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cd3ce-4cb2-49c0-9314-7e46433ee725 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.864018 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cd3ce-4cb2-49c0-9314-7e46433ee725 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.864287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56cd3ce-4cb2-49c0-9314-7e46433ee725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.874082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cd3ce-4cb2-49c0-9314-7e46433ee725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.875522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56cd3ce-4cb2-49c0-9314-7e46433ee725 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:29.884469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 904/1791] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:29 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:29.904891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:29.925567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:29.969231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a9d94-a4f6-4593-a6cf-570c1d0acfff None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:29.974348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:29.974979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:29.975261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:29.975533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] PATH_INFO: `/v3/users/9bf228a4b9144370baf1049347f3a0a1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:29.976661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bf228a4b9144370baf1049347f3a0a1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:29.978612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:30.024827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bf228a4b9144370baf1049347f3a0a1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.025509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bf228a4b9144370baf1049347f3a0a1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.026126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bf228a4b9144370baf1049347f3a0a1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.032961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bf228a4b9144370baf1049347f3a0a1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.040083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bf228a4b9144370baf1049347f3a0a1'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.057370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 888/1792] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:29 2022] DELETE /identity/v3/users/9bf228a4b9144370baf1049347f3a0a1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:30.062766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:30.063097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:30.063368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:30.083974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:30.085197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:30.143997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 905/1793] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:30.154527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:30.155934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:30.211219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690049a1-563f-4afa-833c-a6e869a0446e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:30.214787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f086d-5bd9-435a-b091-209d6fb8608c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:30.216066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f086d-5bd9-435a-b091-209d6fb8608c service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:30.216345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f086d-5bd9-435a-b091-209d6fb8608c service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:30.216613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f086d-5bd9-435a-b091-209d6fb8608c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:30.225439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f086d-5bd9-435a-b091-209d6fb8608c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:30.226858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f086d-5bd9-435a-b091-209d6fb8608c service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:30.236935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 889/1794] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:30.660262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:30.661854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:30.674512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579ec92-6650-4bff-9b33-2b387012c1ad None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:30.679277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:30.679917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:30.680195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:30.681283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] PATH_INFO: `/v3/projects/99bf0889389d4d8f889b61f8ded5471b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:30.682557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bf0889389d4d8f889b61f8ded5471b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:30.685215 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:30.763577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:30.789796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bf0889389d4d8f889b61f8ded5471b', 'request_id': 'req-f53dc68d-4687-4321-a565-b49687f76248'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.798388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bf0889389d4d8f889b61f8ded5471b', 'request_id': 'req-f53dc68d-4687-4321-a565-b49687f76248'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.810086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bf0889389d4d8f889b61f8ded5471b', 'request_id': 'req-f53dc68d-4687-4321-a565-b49687f76248'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.824513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] The token cache is being invalidate because project 99bf0889389d4d8f889b61f8ded5471b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:30.824698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.825640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f53dc68d-4687-4321-a565-b49687f76248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:30.827347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 906/1795] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:30 2022] DELETE /identity/v3/projects/99bf0889389d4d8f889b61f8ded5471b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:30.832374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:30.832826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:30.833012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:30.853600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:30.854651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:30.906839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:30.931140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 890/1796] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:30.939291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:30.939581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:30.939856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:30.959765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:30.960973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:30.969674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 907/1797] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:30.974377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:30.991132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2a93b0-d5f1-4761-81f2-17438eb392bc None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:30.998566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:30.999551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.000008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.000459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.001573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:31.002117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.004979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.024911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 891/1798] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:31.026092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:31.039829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011d304c-9cab-4228-ba14-0cc64e525bb7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:31.046839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:31.048655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.049240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.049769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.050696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.052629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.097347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 908/1799] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:31 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:31.101644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:31.117494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdba5c6b-2d41-44c2-898f-e660d8607437 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:31.181531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:31.182385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.182668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.182941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.183606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.185853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.207019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 892/1800] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:31 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:31.211222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:31.222185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab506e66-0c93-4275-b76a-5857b2a59f1e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:31.226763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:31.227355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.227633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.227903 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.228559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.230193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.240007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 909/1801] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:31.244179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:31.254106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5240f6e-0be5-472c-af1d-4cabd150abc3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:31.258796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4134d-fa3e-4a58-a786-942fac884512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:31.259413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4134d-fa3e-4a58-a786-942fac884512 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.259694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4134d-fa3e-4a58-a786-942fac884512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.259967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4134d-fa3e-4a58-a786-942fac884512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.260620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4134d-fa3e-4a58-a786-942fac884512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.262865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4134d-fa3e-4a58-a786-942fac884512 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.276992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 893/1802] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:31.281921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:31.292378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd02c1d-d8a1-4b3b-bc19-42aa2fff9bb7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:31.297249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:31.297924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.298205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.298479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] PATH_INFO: `/v3/projects/3f584855911f47de84309fc26b2e25ee/users/fd90f4df65dd43e4814a60c1955114fa/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.313031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f584855911f47de84309fc26b2e25ee, user_id=fd90f4df65dd43e4814a60c1955114fa, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.315964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.329197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 910/1803] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:31 2022] PUT /identity/v3/projects/3f584855911f47de84309fc26b2e25ee/users/fd90f4df65dd43e4814a60c1955114fa/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:31.334633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20e1ec-9815-42fd-b0b6-d0400cddf7dc None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.334924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20e1ec-9815-42fd-b0b6-d0400cddf7dc None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.335206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20e1ec-9815-42fd-b0b6-d0400cddf7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.356328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-3f20e1ec-9815-42fd-b0b6-d0400cddf7dc None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:31.357637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3f20e1ec-9815-42fd-b0b6-d0400cddf7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:31.416591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 894/1804] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:31.427050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:31.428769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:31.485932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e204c-a927-4e5d-9d4e-6425789c1fc9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:31.489706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:31.490323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:31.490607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:31.490885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:31.501555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:31.503128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:31.514256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 911/1805] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:32.600495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c1c37-6a52-410e-bea4-d7e249339f76 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:32.600893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c1c37-6a52-410e-bea4-d7e249339f76 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:32.601056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c1c37-6a52-410e-bea4-d7e249339f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:32.634419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-441c1c37-6a52-410e-bea4-d7e249339f76 None None] MFA Rules not processed for user `fd90f4df65dd43e4814a60c1955114fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:32.635940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-441c1c37-6a52-410e-bea4-d7e249339f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:32.676725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:32.712389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 895/1806] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:32.723087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:32.724595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:32.789319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f55c3e-dad4-46b4-93b2-0cf617cfe188 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:32.795859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:32.796465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:32.796744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:32.797010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:32.805376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:32.806802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:32.816005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 912/1807] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:38.501341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763ff14-5ebb-453a-aa99-ff7cf10cb4c1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:38.501923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763ff14-5ebb-453a-aa99-ff7cf10cb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:38.502021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763ff14-5ebb-453a-aa99-ff7cf10cb4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:38.522493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f763ff14-5ebb-453a-aa99-ff7cf10cb4c1 None None] MFA Rules not processed for user `fd90f4df65dd43e4814a60c1955114fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:38.523704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f763ff14-5ebb-453a-aa99-ff7cf10cb4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:38.531169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 896/1808] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:38 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:42.812551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:44.844526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:44.845964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:44.860755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f605777-5b6b-4db7-96ba-c6f10ecd566b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:44.865587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:44.866307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:44.866687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:44.867030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] PATH_INFO: `/v3/users/b95742174c264ffe921eb2c4f5ac5318` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:44.868257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95742174c264ffe921eb2c4f5ac5318)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:44.870621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:44.928931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95742174c264ffe921eb2c4f5ac5318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:44.937676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95742174c264ffe921eb2c4f5ac5318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:44.938496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95742174c264ffe921eb2c4f5ac5318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:44.948467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95742174c264ffe921eb2c4f5ac5318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:44.949250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95742174c264ffe921eb2c4f5ac5318'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:44.970515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 913/1809] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:44 2022] DELETE /identity/v3/users/b95742174c264ffe921eb2c4f5ac5318 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:44.974985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:44.975477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:44.975586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:44.995710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:44.996942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:45.068766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 897/1810] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:45.079450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:45.081728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:45.160826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddfe036-016e-4485-9d74-34f361db771c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:45.170525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:45.171337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:45.171741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:45.172715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:45.183088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:45.184999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:45.199910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 914/1811] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:32:45.599615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:45.601587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:45.617891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3f7bc7-1223-4349-80f8-b2fe0e10688d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:45.625352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:45.626451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:45.627242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:45.627242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] PATH_INFO: `/v3/projects/d5a6e4eaadf74e40a9a9186ccc65f61c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:45.628561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5a6e4eaadf74e40a9a9186ccc65f61c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:45.631007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:45.691398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5a6e4eaadf74e40a9a9186ccc65f61c', 'request_id': 'req-df1538e8-a957-479d-b8dc-8859ba277c8b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.699519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5a6e4eaadf74e40a9a9186ccc65f61c', 'request_id': 'req-df1538e8-a957-479d-b8dc-8859ba277c8b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.707684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5a6e4eaadf74e40a9a9186ccc65f61c', 'request_id': 'req-df1538e8-a957-479d-b8dc-8859ba277c8b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.720401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] The token cache is being invalidate because project d5a6e4eaadf74e40a9a9186ccc65f61c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:45.720790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.721788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.723465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 898/1812] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:45 2022] DELETE /identity/v3/projects/d5a6e4eaadf74e40a9a9186ccc65f61c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:45.727550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:45.737469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74bebf9-db81-4144-95e7-fab23ac3dc17 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:45.802034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:45.802700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:45.802982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:45.803280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] PATH_INFO: `/v3/users/45513bf5b8f24247ad0944988088ad81` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:45.804440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45513bf5b8f24247ad0944988088ad81)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:45.806433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:45.837583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:45.902060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45513bf5b8f24247ad0944988088ad81'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.912310 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45513bf5b8f24247ad0944988088ad81'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.917019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45513bf5b8f24247ad0944988088ad81'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.927120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45513bf5b8f24247ad0944988088ad81'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.927873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04844aef-07fc-4628-951c-bc2c335c7782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45513bf5b8f24247ad0944988088ad81'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:45.948150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 915/1813] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:45 2022] DELETE /identity/v3/users/45513bf5b8f24247ad0944988088ad81 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:46.255103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.256602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:46.280069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1538e8-a957-479d-b8dc-8859ba277c8b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.356540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:46.366713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.367733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.368139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.368667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] PATH_INFO: `/v3/projects/033c56742b12413ebd214130df8a3266` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.369984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] RBAC: Authorizing `identity:delete_project(project_id=033c56742b12413ebd214130df8a3266)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:46.372625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:46.432128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '033c56742b12413ebd214130df8a3266', 'request_id': 'req-c9663e7d-5fc7-45dc-b686-7d8585ead124'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:46.442559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '033c56742b12413ebd214130df8a3266', 'request_id': 'req-c9663e7d-5fc7-45dc-b686-7d8585ead124'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:46.449783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '033c56742b12413ebd214130df8a3266', 'request_id': 'req-c9663e7d-5fc7-45dc-b686-7d8585ead124'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:46.463390 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] The token cache is being invalidate because project 033c56742b12413ebd214130df8a3266 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:46.464199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:46.464484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c9663e7d-5fc7-45dc-b686-7d8585ead124 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:46.471557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 899/1814] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:46 2022] DELETE /identity/v3/projects/033c56742b12413ebd214130df8a3266 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:46.473439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.473730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.474067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.501262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:46.502546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:46.571219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 916/1815] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:46.580594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.580880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.581192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.605127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:46.606679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:46.617268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 900/1816] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:46.621800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.634297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6ba30-eaef-48b4-8ab3-8efcb947a96d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.641571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.642543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.642981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.643753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.644669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:46.645122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:46.647416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:46.666731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 917/1817] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:46.670876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.682078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7145e2-cc6c-4779-af34-ef90e174fa4f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.688026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.688739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.689112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.689525 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.690246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:46.692492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:46.725370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 901/1818] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:46 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:46.729655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.740111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c2a82-a420-42ed-8a89-b43194e802b4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.805723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.806443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.806836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.807167 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.807877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:46.810867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:46.837427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 918/1819] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:46 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:46.840020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.851609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b5cb90-4ea1-4dae-b970-1d7fe3de17db admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.857571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.858362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.859418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.859793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.860596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:46.862314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:46.876219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 902/1820] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:46.881014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.892911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161275ac-e922-42b5-bd25-bdac8f0d5926 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.898131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada7d01-dc60-4ce1-9515-7501101a3a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.898747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7d01-dc60-4ce1-9515-7501101a3a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.899361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7d01-dc60-4ce1-9515-7501101a3a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.899934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada7d01-dc60-4ce1-9515-7501101a3a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:46.900785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada7d01-dc60-4ce1-9515-7501101a3a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:46.903468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada7d01-dc60-4ce1-9515-7501101a3a78 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:46.922851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:46.940325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 919/1821] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:46.943540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:46.960518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc964c-8f23-4f00-b122-0c9512a66122 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:46.966345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:46.967015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:46.967352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:46.967763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] PATH_INFO: `/v3/projects/56897e13ec654ce186b17f270d658c08/users/6b9258f607e64884aab99b7df6602cf7/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:47.006755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56897e13ec654ce186b17f270d658c08, user_id=6b9258f607e64884aab99b7df6602cf7, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:47.010414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:47.060717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 903/1822] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:46 2022] PUT /identity/v3/projects/56897e13ec654ce186b17f270d658c08/users/6b9258f607e64884aab99b7df6602cf7/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:47.063916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:47.064544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:47.065300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:47.159672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] MFA Rules not processed for user `6b9258f607e64884aab99b7df6602cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:47.162203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:47.293106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 920/1823] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:47.308449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:47.310464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:47.397443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d79c4-cc01-4c92-a1c4-3dc7d59e5813 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:47.401450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:47.402268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:47.402697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:47.403115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:47.414522 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:47.416308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:47.430041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 904/1824] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:49.924383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:49.925965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:50.021736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748f810-8617-437a-8111-2e9a495c9e3a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.026229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.026857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.027134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.027406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] PATH_INFO: `/v3/users/25119286009743bbac4217d5faa40177` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.028513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25119286009743bbac4217d5faa40177)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.031146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.084734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25119286009743bbac4217d5faa40177'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.093958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25119286009743bbac4217d5faa40177'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.100430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25119286009743bbac4217d5faa40177'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.113336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25119286009743bbac4217d5faa40177'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.114063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d059251b-cb7c-493a-8575-8c8e5361fa71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25119286009743bbac4217d5faa40177'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.131795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 921/1825] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:49 2022] DELETE /identity/v3/users/25119286009743bbac4217d5faa40177 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:50.305972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.327367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a4659-97e1-4440-8e34-348fa4f5c5a4 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.352648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:50.390504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.391137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.391414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.391684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] PATH_INFO: `/v3/projects/c899460f7a66440b9321189597303fc9` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.392790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c899460f7a66440b9321189597303fc9)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.394726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.441873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c899460f7a66440b9321189597303fc9', 'request_id': 'req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.448983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c899460f7a66440b9321189597303fc9', 'request_id': 'req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.455527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c899460f7a66440b9321189597303fc9', 'request_id': 'req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.465791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] The token cache is being invalidate because project c899460f7a66440b9321189597303fc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:50.466083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.466760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7df22f7f-6c28-4ce3-af36-8f3c2fcfb665 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:50.468757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 905/1826] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:50 2022] DELETE /identity/v3/projects/c899460f7a66440b9321189597303fc9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:50.473556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.473843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.474118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.494439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:50.495656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:50.499853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:50.555462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 922/1827] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:50.563170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.563450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.563717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.583909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:50.585116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:50.593602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 906/1828] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:50.597658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.607335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec495e30-3ebd-4159-af8a-ecea4008c20f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.612042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.612656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.612931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.613198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.613874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:50.614170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.615756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.630179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 923/1829] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:32:50.634887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.646108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d937db-e133-45c8-a58b-170333e09a2d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.650873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.651466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.651743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.652011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.652631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.654597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.681581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 907/1830] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:50 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:50.685789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.695900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d47a0-6338-4b4b-8518-8aa9b4176d20 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.751546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.752129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.752411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.752684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.753345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.755266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.776668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 924/1831] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:50 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:50.781860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.791854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295c866-3fe2-4fa3-8740-077bffd52d9a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.796721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.797337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.797611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.797882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.798511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.800121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.809847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 908/1832] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:50.813892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.823716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84aa8fc-13c9-4224-934e-9a4c62a5b995 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.828774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6237298-17a5-4e42-b400-d1b1fa9d0bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.829391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6237298-17a5-4e42-b400-d1b1fa9d0bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.829673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6237298-17a5-4e42-b400-d1b1fa9d0bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.829944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6237298-17a5-4e42-b400-d1b1fa9d0bb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.830574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6237298-17a5-4e42-b400-d1b1fa9d0bb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.832153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6237298-17a5-4e42-b400-d1b1fa9d0bb0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.842556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 925/1833] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:50.847330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:50.856922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75ecd5c-5f3a-43ac-ae21-a39a1b691a8a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:50.861738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:50.862371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.862644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.862913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] PATH_INFO: `/v3/projects/d0bc16bf77c84b1ea38b544bd8b889a9/users/4e0ea10c39bd4813a36edd37914df98e/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.878306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bc16bf77c84b1ea38b544bd8b889a9, user_id=4e0ea10c39bd4813a36edd37914df98e, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:50.881122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:50.894983 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 909/1834] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:50 2022] PUT /identity/v3/projects/d0bc16bf77c84b1ea38b544bd8b889a9/users/4e0ea10c39bd4813a36edd37914df98e/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:50.906583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:50.906867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:50.907142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:50.940441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] MFA Rules not processed for user `4e0ea10c39bd4813a36edd37914df98e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:50.941718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:51.017665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 926/1835] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:50 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:51.022257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.087157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:51.101337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a750de4-3631-4502-9c47-988d34ab0361 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.106236 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.106856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.107141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.107417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.108067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.110076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.144900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 910/1836] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:51 2022] POST /identity/v3/projects => generated 405 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:51.145905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.158904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:51.163721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffba9f8-a02a-4e55-aa3b-38987c1df45b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.223727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.224482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.224763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.225031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.225696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.227603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.252183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 927/1837] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:51 2022] POST /identity/v3/users => generated 309 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:51.253338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.263572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9678e0-f600-4ff9-bf45-2d16de27b2d1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.274860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.275480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.275758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.276067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.276717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.278534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.289913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 911/1838] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:51.294058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.304651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0230e07b-1d58-494a-b88a-30b0e3b3d87e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.309279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.309962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.310242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.310512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] PATH_INFO: `/v3/projects/960f1431ed8e4f92ba91f51081d7da7f/users/2029ffdc91c44802bcd34dbd898ecdbd/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.326071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=960f1431ed8e4f92ba91f51081d7da7f, user_id=2029ffdc91c44802bcd34dbd898ecdbd, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.328850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.343524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 928/1839] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:51 2022] PUT /identity/v3/projects/960f1431ed8e4f92ba91f51081d7da7f/users/2029ffdc91c44802bcd34dbd898ecdbd/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:51.347429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.365771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d87ee-63d6-4d67-b721-11a9464fe0e4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.422384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b063ae6d-5bc8-4e21-9658-61834ee29ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.422980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063ae6d-5bc8-4e21-9658-61834ee29ca6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.423256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063ae6d-5bc8-4e21-9658-61834ee29ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.423524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b063ae6d-5bc8-4e21-9658-61834ee29ca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.424155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063ae6d-5bc8-4e21-9658-61834ee29ca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.425764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b063ae6d-5bc8-4e21-9658-61834ee29ca6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.435189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 912/1840] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:51.440276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.450261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44843684-6e32-4710-86c8-25d06a561db0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.455042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.455680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.455955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.456228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] PATH_INFO: `/v3/projects/960f1431ed8e4f92ba91f51081d7da7f/users/2029ffdc91c44802bcd34dbd898ecdbd/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.458101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] RBAC: Authorizing `identity:create_grant(project_id=960f1431ed8e4f92ba91f51081d7da7f, user_id=2029ffdc91c44802bcd34dbd898ecdbd, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.460712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.470902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 929/1841] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:51 2022] PUT /identity/v3/projects/960f1431ed8e4f92ba91f51081d7da7f/users/2029ffdc91c44802bcd34dbd898ecdbd/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:51.486735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.486948 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.487216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.516205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] MFA Rules not processed for user `2029ffdc91c44802bcd34dbd898ecdbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:51.517420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:51.583576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 913/1842] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:51 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:51.594477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.596187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:51.653172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fd4016-64fb-4d87-b5fb-6a14191a415e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.657723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c5a296-7791-4aad-9eab-4b433d23cfff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.658114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a296-7791-4aad-9eab-4b433d23cfff service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.658395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a296-7791-4aad-9eab-4b433d23cfff service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.658668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c5a296-7791-4aad-9eab-4b433d23cfff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.667531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5a296-7791-4aad-9eab-4b433d23cfff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.668948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c5a296-7791-4aad-9eab-4b433d23cfff service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:51.678899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 930/1843] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:51 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:32:51.908884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:51.955439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:51.970982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f05973-3829-4a4d-bd0a-d48b7b08e80a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:51.975487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:51.976110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:51.976392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:51.976666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] PATH_INFO: `/v3/users/4e0ea10c39bd4813a36edd37914df98e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:51.977813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e0ea10c39bd4813a36edd37914df98e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:51.979742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:52.025870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e0ea10c39bd4813a36edd37914df98e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.026450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e0ea10c39bd4813a36edd37914df98e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.027054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e0ea10c39bd4813a36edd37914df98e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.034460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e0ea10c39bd4813a36edd37914df98e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.041579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e0ea10c39bd4813a36edd37914df98e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.059350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 914/1844] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:51 2022] DELETE /identity/v3/users/4e0ea10c39bd4813a36edd37914df98e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:52.064075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb328484-372c-4e93-bada-898ee44831ef None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:52.064526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb328484-372c-4e93-bada-898ee44831ef None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:52.064619 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb328484-372c-4e93-bada-898ee44831ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:52.083965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-cb328484-372c-4e93-bada-898ee44831ef None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:52.085181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cb328484-372c-4e93-bada-898ee44831ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:52.136181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:52.146258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 931/1845] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:32:52.156572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:52.158064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:52.217012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac2f59-825b-4761-8239-df2250a5ad6f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:52.222102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:52.222710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:52.223161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:52.223284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:52.235420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:52.237139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:52.248245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 915/1846] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:52.623481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb328484-372c-4e93-bada-898ee44831ef None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:52.625736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-cb328484-372c-4e93-bada-898ee44831ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:52.637959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb328484-372c-4e93-bada-898ee44831ef None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:52.643561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:52.644198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:52.644503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:52.644777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] PATH_INFO: `/v3/projects/d0bc16bf77c84b1ea38b544bd8b889a9` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:52.645920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0bc16bf77c84b1ea38b544bd8b889a9)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:52.647947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:52.693828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0bc16bf77c84b1ea38b544bd8b889a9', 'request_id': 'req-8339fbf6-937b-4313-9d2b-cdffd272a986'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.700019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0bc16bf77c84b1ea38b544bd8b889a9', 'request_id': 'req-8339fbf6-937b-4313-9d2b-cdffd272a986'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.707288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0bc16bf77c84b1ea38b544bd8b889a9', 'request_id': 'req-8339fbf6-937b-4313-9d2b-cdffd272a986'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.717683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] The token cache is being invalidate because project d0bc16bf77c84b1ea38b544bd8b889a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:52.718127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.718887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.720369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 932/1847] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:52 2022] DELETE /identity/v3/projects/d0bc16bf77c84b1ea38b544bd8b889a9 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:52.724427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:52.734601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d68c55a-fd86-4908-96ec-0d3175c4b059 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:52.792633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:52.793172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:52.793484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:52.793757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] PATH_INFO: `/v3/users/2029ffdc91c44802bcd34dbd898ecdbd` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:52.794868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2029ffdc91c44802bcd34dbd898ecdbd)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:52.797352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:52.807167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:52.843169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2029ffdc91c44802bcd34dbd898ecdbd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.843810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2029ffdc91c44802bcd34dbd898ecdbd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.844432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2029ffdc91c44802bcd34dbd898ecdbd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.851153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2029ffdc91c44802bcd34dbd898ecdbd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.857979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e3ebcba3-e8b7-4600-9e11-cf96c7a3a32a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2029ffdc91c44802bcd34dbd898ecdbd'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:52.876434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 916/1848] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:52 2022] DELETE /identity/v3/users/2029ffdc91c44802bcd34dbd898ecdbd => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:53.139212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.140844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:53.154206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339fbf6-937b-4313-9d2b-cdffd272a986 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.212212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.212911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.213132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.213400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] PATH_INFO: `/v3/projects/960f1431ed8e4f92ba91f51081d7da7f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.214509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=960f1431ed8e4f92ba91f51081d7da7f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.216390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.262036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960f1431ed8e4f92ba91f51081d7da7f', 'request_id': 'req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:53.268460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960f1431ed8e4f92ba91f51081d7da7f', 'request_id': 'req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:53.276152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960f1431ed8e4f92ba91f51081d7da7f', 'request_id': 'req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:53.287127 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] The token cache is being invalidate because project 960f1431ed8e4f92ba91f51081d7da7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:53.287419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:53.288050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a2f3b09f-d620-4c04-ab21-ff208c7e9a3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:53.289570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 933/1849] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:53 2022] DELETE /identity/v3/projects/960f1431ed8e4f92ba91f51081d7da7f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:53.294323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107728f3-a39f-443b-8413-98c29dda986a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.294634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107728f3-a39f-443b-8413-98c29dda986a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.294905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107728f3-a39f-443b-8413-98c29dda986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.314698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-107728f3-a39f-443b-8413-98c29dda986a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:53.315815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-107728f3-a39f-443b-8413-98c29dda986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:53.377080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 917/1850] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:53.384732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.385019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.385330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.404770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:53.406520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:53.414832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 934/1851] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:53.418860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107728f3-a39f-443b-8413-98c29dda986a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.428940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107728f3-a39f-443b-8413-98c29dda986a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.434901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.435800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.436084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.436358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.437165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:53.437593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.439207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.451265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 918/1852] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:53.455811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.465508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11af84c-1bc3-4923-8828-02a99e01a8c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.471272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.472098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.472948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.473869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.474932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.477280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.482164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:53.519186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 935/1853] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:53.523327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.534117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033f88d-96da-456f-8285-89e23fee31cc admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.595783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.596401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.596684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.596959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.597695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.599605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.622657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 919/1854] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:53.627526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.637779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bef53a-470f-48a2-93d6-25c6737e82f3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.643464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.644063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.644342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.644629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.645252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.646885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.657060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 936/1855] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:53.661159 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.667524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:53.672471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604cc83-f179-4e24-9965-d4f527e6534a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.678801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a979f1-26a0-41b2-ad7b-4470bab1b249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.679594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a979f1-26a0-41b2-ad7b-4470bab1b249 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.680037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a979f1-26a0-41b2-ad7b-4470bab1b249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.680473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a979f1-26a0-41b2-ad7b-4470bab1b249 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.681117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a979f1-26a0-41b2-ad7b-4470bab1b249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.682835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a979f1-26a0-41b2-ad7b-4470bab1b249 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.694954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 920/1856] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:53.699138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.709030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af848dbe-00d3-4892-bd98-63c078b5f6f1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.713757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.714410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.714687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.714961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] PATH_INFO: `/v3/projects/539c773838184a75a66623c40460c79d/users/6d8c1b26ce714ac69a0042297f0b6359/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.730254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539c773838184a75a66623c40460c79d, user_id=6d8c1b26ce714ac69a0042297f0b6359, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.732915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.746283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 937/1857] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:53 2022] PUT /identity/v3/projects/539c773838184a75a66623c40460c79d/users/6d8c1b26ce714ac69a0042297f0b6359/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:53.758354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.758828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.758920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.786046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] MFA Rules not processed for user `6d8c1b26ce714ac69a0042297f0b6359`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:53.787248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:53.859046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 921/1858] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:53.863576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.930955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a5a258-8b9d-48b1-9b3b-5cd96a85a4d1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:53.935800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:53.936404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:53.936839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:53.937193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:53.937855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:53.939768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:53.969969 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 938/1859] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:53.974041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:53.986220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b5c054-99cc-474a-8637-173f98e4007f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.045266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.046541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.046914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.047184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.047805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.049853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.072738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 922/1860] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:53 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:54.076743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.087192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce79ce-ac4d-4ca2-854c-9335c93d1f56 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.091687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.092287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.092562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.093479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.094398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.095972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.106402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 939/1861] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:54.111207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.121227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a652129-c819-4d65-aa8f-8267abee66c2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.126086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.126732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.127006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.127276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] PATH_INFO: `/v3/projects/abed8de91fa344afafbc82c7059650d7/users/689826bc153c408c845b32638410627d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.142313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=abed8de91fa344afafbc82c7059650d7, user_id=689826bc153c408c845b32638410627d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.144958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.157659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 923/1862] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:54 2022] PUT /identity/v3/projects/abed8de91fa344afafbc82c7059650d7/users/689826bc153c408c845b32638410627d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:54.161584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.172319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6d0c-c4cf-40ae-9bf1-33c2729b4bfb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.238260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0944bf7-fc66-4834-b877-e6d9ca1b563a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.238860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944bf7-fc66-4834-b877-e6d9ca1b563a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.239140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944bf7-fc66-4834-b877-e6d9ca1b563a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.239412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0944bf7-fc66-4834-b877-e6d9ca1b563a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.240047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0944bf7-fc66-4834-b877-e6d9ca1b563a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.241784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0944bf7-fc66-4834-b877-e6d9ca1b563a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.252722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 940/1863] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:54.256880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.272906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8294bf1-3369-423f-ad6c-a0e2bd6d84dd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.277754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.278405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.278680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.278950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] PATH_INFO: `/v3/projects/abed8de91fa344afafbc82c7059650d7/users/689826bc153c408c845b32638410627d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.281456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abed8de91fa344afafbc82c7059650d7, user_id=689826bc153c408c845b32638410627d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.284149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.296297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 924/1864] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:54 2022] PUT /identity/v3/projects/abed8de91fa344afafbc82c7059650d7/users/689826bc153c408c845b32638410627d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:54.307561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.307860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.308157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.341094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] MFA Rules not processed for user `689826bc153c408c845b32638410627d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:54.342685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:54.351534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:54.423631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 941/1865] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:54 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:54.435344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.438027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:54.480962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:54.501580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace36de0-973c-4ad4-aaa7-e934aa7bcb01 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.505392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.506005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.506283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.506557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.515918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.517384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.530296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 925/1866] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:54 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:54.862067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.871379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f446b381-1571-4312-86fc-245cdf54c656 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.876285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689826bc153c408c845b32638410627d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abed8de91fa344afafbc82c7059650d7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.876888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.877180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.877487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.878438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:54.878438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.880466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.891654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 942/1867] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:54.895659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.904708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d7d28-9ca0-4e6a-ba45-87863f62f39f service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:54.910189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689826bc153c408c845b32638410627d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abed8de91fa344afafbc82c7059650d7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:54.911067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:54.911171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:54.911369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:54.912144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:54.914081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:54.940862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 926/1868] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:54 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:54.945103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:54.957381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665d921-5342-47e1-91a4-ae5ed2edba77 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:55.013208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689826bc153c408c845b32638410627d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abed8de91fa344afafbc82c7059650d7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:55.013850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.014134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.014415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.015967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:55.015967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:55.017647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:55.034474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 943/1869] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:55.069408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:55.080051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6d2be-7045-4abb-8445-bde4f01248a4 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:55.084552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '689826bc153c408c845b32638410627d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'abed8de91fa344afafbc82c7059650d7', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:55.085089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.085401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.085672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] PATH_INFO: `/v3/projects/6b57452bf7a54458bb0bdffbe9329768` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.086768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6b57452bf7a54458bb0bdffbe9329768)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:55.088692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:55.134372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b57452bf7a54458bb0bdffbe9329768', 'request_id': 'req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.142153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b57452bf7a54458bb0bdffbe9329768', 'request_id': 'req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.148899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b57452bf7a54458bb0bdffbe9329768', 'request_id': 'req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.159837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] The token cache is being invalidate because project 6b57452bf7a54458bb0bdffbe9329768 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:55.160128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.160801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a1ab8de2-9c58-417c-8a8f-1cf3a21ef5b3 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.162393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 927/1870] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:55 2022] DELETE /identity/v3/projects/6b57452bf7a54458bb0bdffbe9329768 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:55.347859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:55.349765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:55.413873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abadeea-8fb8-4365-9f27-1457265d73d7 tempest-VolumeQuotaClassesTest-699805454 tempest-VolumeQuotaClassesTest-699805454-project-admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:55.443154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:55.471268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:55.472022 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.472336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.472611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] PATH_INFO: `/v3/users/6d8c1b26ce714ac69a0042297f0b6359` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.473768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d8c1b26ce714ac69a0042297f0b6359)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:55.476078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:55.522133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d8c1b26ce714ac69a0042297f0b6359'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.530600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d8c1b26ce714ac69a0042297f0b6359'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.531271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d8c1b26ce714ac69a0042297f0b6359'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.538366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d8c1b26ce714ac69a0042297f0b6359'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.538989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d8c1b26ce714ac69a0042297f0b6359'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:55.558632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 944/1871] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:55 2022] DELETE /identity/v3/users/6d8c1b26ce714ac69a0042297f0b6359 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:55.563617 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.563898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.564172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.583660 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:55.584852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:55.644866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 928/1872] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:55.654371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:55.656504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:55.713720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e15710-6df8-4c82-8fe5-1fda08e5401a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:55.717292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:55.718454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:55.718727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:55.718989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:55.727852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:55.729271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:55.739089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 945/1873] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:56.124941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:56.126571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:56.144946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68791-8966-4568-86cf-e1a894bc931f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:56.156677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:56.157342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.157632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.158009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] PATH_INFO: `/v3/projects/539c773838184a75a66623c40460c79d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.159169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539c773838184a75a66623c40460c79d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:56.161110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:56.198469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:56.209996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539c773838184a75a66623c40460c79d', 'request_id': 'req-ac96f204-495a-41c8-8b01-b23a52e8c2e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.216882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539c773838184a75a66623c40460c79d', 'request_id': 'req-ac96f204-495a-41c8-8b01-b23a52e8c2e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.223946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539c773838184a75a66623c40460c79d', 'request_id': 'req-ac96f204-495a-41c8-8b01-b23a52e8c2e4'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.235318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] The token cache is being invalidate because project 539c773838184a75a66623c40460c79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:56.235607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.236247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.237767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 929/1874] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:56 2022] DELETE /identity/v3/projects/539c773838184a75a66623c40460c79d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:56.241693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:56.251554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e801af-3740-421f-86b6-4e54d987cb87 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:56.301244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:56.309150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:56.310134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.310453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.310729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] PATH_INFO: `/v3/users/689826bc153c408c845b32638410627d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.312547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=689826bc153c408c845b32638410627d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:56.314700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:56.362813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '689826bc153c408c845b32638410627d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.370273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '689826bc153c408c845b32638410627d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.371323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '689826bc153c408c845b32638410627d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.378780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '689826bc153c408c845b32638410627d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.378780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-fde533ce-a271-4b39-a77b-278a27dafa05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '689826bc153c408c845b32638410627d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.395593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 946/1875] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:32:56 2022] DELETE /identity/v3/users/689826bc153c408c845b32638410627d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:56.660538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:56.662991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:56.677765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac96f204-495a-41c8-8b01-b23a52e8c2e4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:56.735188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:56.735815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.736091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.736359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] PATH_INFO: `/v3/projects/abed8de91fa344afafbc82c7059650d7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.737487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] RBAC: Authorizing `identity:delete_project(project_id=abed8de91fa344afafbc82c7059650d7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:56.739398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:56.784786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abed8de91fa344afafbc82c7059650d7', 'request_id': 'req-59e36d02-4e23-4158-a96c-059738bcb12d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.791688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abed8de91fa344afafbc82c7059650d7', 'request_id': 'req-59e36d02-4e23-4158-a96c-059738bcb12d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.798015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abed8de91fa344afafbc82c7059650d7', 'request_id': 'req-59e36d02-4e23-4158-a96c-059738bcb12d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.807923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] The token cache is being invalidate because project abed8de91fa344afafbc82c7059650d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:32:56.808206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.808830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59e36d02-4e23-4158-a96c-059738bcb12d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:32:56.810355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 930/1876] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:32:56 2022] DELETE /identity/v3/projects/abed8de91fa344afafbc82c7059650d7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:56.815263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.815610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.815893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.836339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:56.837579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:56.897346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 947/1877] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:56.904959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.905288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.905782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.926799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:56.927990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:56.935822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 931/1878] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:56.940433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:56.950204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f88cdb9-60e4-4885-aa90-0d0af2c7ead6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:56.955534 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:56.956128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.956408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.956679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.957351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:32:56.957654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:56.959246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:56.970689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 948/1879] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:32:56.974812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:56.983971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a8342b-db69-4dd1-a0c6-5a4432e2a129 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:56.988509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:56.989097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:56.989400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:56.989670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:56.990299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:56.992227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.025285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 932/1880] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:56 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:57.026420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.036721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e465f-53e6-483c-be91-2452a7d0f423 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.097833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.098448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.098727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.098997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.099671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.101665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.111464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:57.129446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 949/1881] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:57 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:32:57.130682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.144083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e995e-8950-4e77-b02c-29de461996cb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.153531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.154196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.154532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.154835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.155629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.157245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.168254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 933/1882] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:57.172312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.182654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f757f-c6ad-4e94-80f2-66181d1c2376 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.188237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c343ecb6-132f-4a2a-88f2-151a9e99088e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.188850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343ecb6-132f-4a2a-88f2-151a9e99088e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.189130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343ecb6-132f-4a2a-88f2-151a9e99088e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.189558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c343ecb6-132f-4a2a-88f2-151a9e99088e admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.190203 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343ecb6-132f-4a2a-88f2-151a9e99088e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.191807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c343ecb6-132f-4a2a-88f2-151a9e99088e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.204035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 950/1883] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:32:57.207949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.218134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b939c2-1d66-4eca-9f53-602ab555a756 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.223021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.223905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.224190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.224464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] PATH_INFO: `/v3/projects/5034f35b9fdc45c49e9c8230c19893e7/users/094364b4f93944e1a5fef3981bc930f7/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.236234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:57.241161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5034f35b9fdc45c49e9c8230c19893e7, user_id=094364b4f93944e1a5fef3981bc930f7, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.244320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.263032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 934/1884] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:57 2022] PUT /identity/v3/projects/5034f35b9fdc45c49e9c8230c19893e7/users/094364b4f93944e1a5fef3981bc930f7/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:32:57.276031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.276353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.276591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.306694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] MFA Rules not processed for user `094364b4f93944e1a5fef3981bc930f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:57.307908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:57.375813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 951/1885] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:57.380267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.440934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a041bc-a727-4a51-aceb-7f558ce69bf8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.445579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.446209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.446489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.446762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.447424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.449359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.475269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 935/1886] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:32:57 2022] POST /identity/v3/projects => generated 393 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:57.479232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.490302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41d805e-3e87-472e-9d58-c7383d8407e3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.551292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.551893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.552169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.552444 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.553077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.555056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.576561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 952/1887] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:32:57 2022] POST /identity/v3/users => generated 303 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:32:57.580750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.590955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826dd09-103f-4573-adc0-70c920f1f7a9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.595695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.596286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.596566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.596836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.597504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.599090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.610077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 936/1888] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:57.614148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.624972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb789dd5-2218-4b84-9c0a-bdaf5aeac2af admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.629662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.630305 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.630586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.630861 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] PATH_INFO: `/v3/projects/18cb836ebac24ceca14eee3849c0d035/users/3cf191f6053e4e1088b0f880df47b07d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.645492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cb836ebac24ceca14eee3849c0d035, user_id=3cf191f6053e4e1088b0f880df47b07d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.648153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.661912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 953/1889] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:57 2022] PUT /identity/v3/projects/18cb836ebac24ceca14eee3849c0d035/users/3cf191f6053e4e1088b0f880df47b07d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:57.665737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.676711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330ddc4-006f-47f2-bb0a-95bbc4caaa4a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.735487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c82ee-419c-4690-b020-b1cdb5251e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.736098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c82ee-419c-4690-b020-b1cdb5251e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.736369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c82ee-419c-4690-b020-b1cdb5251e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.736642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c82ee-419c-4690-b020-b1cdb5251e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.737283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c82ee-419c-4690-b020-b1cdb5251e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.738907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c82ee-419c-4690-b020-b1cdb5251e2d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.752532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 937/1890] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:32:57.753675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.763844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34eaa08-38a9-4775-be51-003c081a39e7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.769107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.769799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.770081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.770361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] PATH_INFO: `/v3/projects/18cb836ebac24ceca14eee3849c0d035/users/3cf191f6053e4e1088b0f880df47b07d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.772243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18cb836ebac24ceca14eee3849c0d035, user_id=3cf191f6053e4e1088b0f880df47b07d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.774932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.785435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 954/1891] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:32:57 2022] PUT /identity/v3/projects/18cb836ebac24ceca14eee3849c0d035/users/3cf191f6053e4e1088b0f880df47b07d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:32:57.795410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.795701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.796043 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.824163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] MFA Rules not processed for user `3cf191f6053e4e1088b0f880df47b07d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:32:57.825501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:57.871883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:57.895538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 938/1892] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:32:57 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:32:57.906376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:57.908076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:57.934116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:32:57.967932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783fa8ae-d9b3-4879-86d9-c0c2c316276c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:57.971689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:57.972290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:57.972569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:57.972839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:57.981672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:57.983101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:57.992347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 955/1893] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:57 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:32:58.038044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:32:58.039430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:32:58.051715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57029b-4fa2-4263-b202-f632cffdc138 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:32:58.055251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b569ff-608f-4305-85d9-aa50cf41fc04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:32:58.055856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b569ff-608f-4305-85d9-aa50cf41fc04 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:32:58.056116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b569ff-608f-4305-85d9-aa50cf41fc04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:32:58.056385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b569ff-608f-4305-85d9-aa50cf41fc04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:32:58.112693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b569ff-608f-4305-85d9-aa50cf41fc04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:32:58.114192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b569ff-608f-4305-85d9-aa50cf41fc04 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:32:58.123639 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 939/1894] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:32:58 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:07.992455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=16 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:08.119376 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=35 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:08.648796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:08.650233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.716731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6a39b-45dd-4840-8483-053ef1389022 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:08.721382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:08.722009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:08.722290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:08.722567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] PATH_INFO: `/v3/users/6b9258f607e64884aab99b7df6602cf7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:08.723844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b9258f607e64884aab99b7df6602cf7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:08.725796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:08.775475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b9258f607e64884aab99b7df6602cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:08.787654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b9258f607e64884aab99b7df6602cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:08.788431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b9258f607e64884aab99b7df6602cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:08.799697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b9258f607e64884aab99b7df6602cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:08.800620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b9258f607e64884aab99b7df6602cf7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:08.823883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 956/1895] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:08 2022] DELETE /identity/v3/users/6b9258f607e64884aab99b7df6602cf7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:08.828756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:08.829040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:08.829795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:08.851041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:08.852249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.869797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:08.871206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.931232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 940/1896] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:08.943850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:08.945946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:08.962093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbacda08-d0e7-458f-a9d2-5e944d29b187 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:08.977101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:08.979038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:08.979038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:08.979213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] PATH_INFO: `/v3/users/094364b4f93944e1a5fef3981bc930f7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:08.981583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] RBAC: Authorizing `identity:delete_user(user_id=094364b4f93944e1a5fef3981bc930f7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:08.984302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.040720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:09.074065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8660cb1d-2ba6-483b-a71e-e3d9361ef3da None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:09.083725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81f041-b117-4d3b-994f-21f490131687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:09.084544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81f041-b117-4d3b-994f-21f490131687 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.084828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81f041-b117-4d3b-994f-21f490131687 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.085100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81f041-b117-4d3b-994f-21f490131687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.085870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '094364b4f93944e1a5fef3981bc930f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.097164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '094364b4f93944e1a5fef3981bc930f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.098522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '094364b4f93944e1a5fef3981bc930f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.099598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81f041-b117-4d3b-994f-21f490131687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:09.101209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81f041-b117-4d3b-994f-21f490131687 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.107863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '094364b4f93944e1a5fef3981bc930f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.109464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '094364b4f93944e1a5fef3981bc930f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.115997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 941/1897] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:33:09.130374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 957/1898] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:08 2022] DELETE /identity/v3/users/094364b4f93944e1a5fef3981bc930f7 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:33:09.134830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.135318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.135421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.165822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:09.167194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:09.256721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 942/1899] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:09.269344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:09.271066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:09.351066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9873bc-effe-4ca9-9fb2-1b375117dcff admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:09.355363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb66db2-fb14-4385-b62c-f7269fb9eecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:09.356023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb66db2-fb14-4385-b62c-f7269fb9eecc service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.356293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb66db2-fb14-4385-b62c-f7269fb9eecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.357205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb66db2-fb14-4385-b62c-f7269fb9eecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.369858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb66db2-fb14-4385-b62c-f7269fb9eecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:09.371321 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb66db2-fb14-4385-b62c-f7269fb9eecc service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.384118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 958/1900] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:09.590930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:09.592360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:09.606412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7f25f-a251-4b84-a208-8096396f5a09 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:09.613005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:09.613731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.614494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.614956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] PATH_INFO: `/v3/projects/56897e13ec654ce186b17f270d658c08` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.616336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=56897e13ec654ce186b17f270d658c08)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:09.618312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.666425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:09.676476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56897e13ec654ce186b17f270d658c08', 'request_id': 'req-313a5239-58ef-454a-bff3-7aeac920f6ae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.690433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56897e13ec654ce186b17f270d658c08', 'request_id': 'req-313a5239-58ef-454a-bff3-7aeac920f6ae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.699841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56897e13ec654ce186b17f270d658c08', 'request_id': 'req-313a5239-58ef-454a-bff3-7aeac920f6ae'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.711207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] The token cache is being invalidate because project 56897e13ec654ce186b17f270d658c08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:09.711517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.712173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.714127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 943/1901] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:09 2022] DELETE /identity/v3/projects/56897e13ec654ce186b17f270d658c08 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:09.719705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1bed-a0b7-4d6c-b9f3-587069f603c7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.719989 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1bed-a0b7-4d6c-b9f3-587069f603c7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.720262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e1bed-a0b7-4d6c-b9f3-587069f603c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.741337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ae3e1bed-a0b7-4d6c-b9f3-587069f603c7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:09.742582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae3e1bed-a0b7-4d6c-b9f3-587069f603c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:09.787106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:09.788586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:09.813209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 959/1902] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:09.822229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.822922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.822922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.846885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:09.848100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:09.857922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 960/1903] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:09.863713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:09.872269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038b23b-98d8-43f9-82bc-e2ccb3ac715d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:09.877518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:09.878132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.878418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.878675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.879318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:09.879615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:09.881212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.897946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a5239-58ef-454a-bff3-7aeac920f6ae admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:09.904607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:09.905695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.906001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.906394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] PATH_INFO: `/v3/projects/5034f35b9fdc45c49e9c8230c19893e7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.908166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5034f35b9fdc45c49e9c8230c19893e7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:09.910728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.922717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 961/1904] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:09.933757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:09.954384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d46190-0265-4fcb-ac53-854d34c7e3aa admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:09.959232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:09.959841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:09.960125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:09.960402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:09.961040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:09.962990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:09.973936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5034f35b9fdc45c49e9c8230c19893e7', 'request_id': 'req-f7838a83-de78-42d1-b08b-71157589297d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.981926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5034f35b9fdc45c49e9c8230c19893e7', 'request_id': 'req-f7838a83-de78-42d1-b08b-71157589297d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.984633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:09.989606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5034f35b9fdc45c49e9c8230c19893e7', 'request_id': 'req-f7838a83-de78-42d1-b08b-71157589297d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:09.997236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 962/1905] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:09 2022] POST /identity/v3/projects => generated 361 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:09.998112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.003800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] The token cache is being invalidate because project 5034f35b9fdc45c49e9c8230c19893e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:10.004546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.005027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.006930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 944/1906] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:09 2022] DELETE /identity/v3/projects/5034f35b9fdc45c49e9c8230c19893e7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:33:10.010689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.012037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:10.086391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02269aa-e3f2-4848-8d73-de991629291e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.091667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.092407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.092878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.093408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.094048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.095980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.101368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:10.105089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7838a83-de78-42d1-b08b-71157589297d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.117262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.118400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.118943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.119234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] PATH_INFO: `/v3/users/3cf191f6053e4e1088b0f880df47b07d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.120382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cf191f6053e4e1088b0f880df47b07d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.122387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.126797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 963/1907] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:09 2022] POST /identity/v3/users => generated 281 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:10.128066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.130027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:10.146656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78a3a93-51d6-47f3-9cfb-9301e9f54cd6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.153732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.154528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.154814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.155366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.156007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.159550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.172372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 964/1908] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:10.176255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.185176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cf191f6053e4e1088b0f880df47b07d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.187119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ceb3ca-28ed-4cdc-9a2d-899d61d06a69 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.189793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cf191f6053e4e1088b0f880df47b07d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.190668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cf191f6053e4e1088b0f880df47b07d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.191814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.192553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.192837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.193245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.193926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.195551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.200899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cf191f6053e4e1088b0f880df47b07d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.210902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 965/1909] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:33:10.211948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.212355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-8920778a-ccc8-4c79-afb4-71e2ed5e556b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cf191f6053e4e1088b0f880df47b07d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:10.214778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:10.230673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0be0a5e-20ce-4f54-935d-a64381e363fb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.236013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 945/1910] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:10 2022] DELETE /identity/v3/users/3cf191f6053e4e1088b0f880df47b07d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:33:10.238954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.240021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.240308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.240588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] PATH_INFO: `/v3/projects/0d3c098149e044b38565858580112f75/users/dba218265e1146c786a2b6c6adfb56a1/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.274267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d3c098149e044b38565858580112f75, user_id=dba218265e1146c786a2b6c6adfb56a1, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.277629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.295570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 966/1911] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:10 2022] PUT /identity/v3/projects/0d3c098149e044b38565858580112f75/users/dba218265e1146c786a2b6c6adfb56a1/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:10.305801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.306082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.306427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.367258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] MFA Rules not processed for user `dba218265e1146c786a2b6c6adfb56a1`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:10.368483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:10.525201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 946/1912] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:10.734896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.736616 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:10.770562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.772325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:10.834159 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5754d84-275c-4104-a6cf-c59d4b425e54 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.839982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.841326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.841625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.841885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.854567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.856654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.868936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 967/1913] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:10.886918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5f6bf-9d2e-45c0-898f-8ba247924dfb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.890352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:10.893826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.894574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.894864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.895138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] PATH_INFO: `/v3/projects/18cb836ebac24ceca14eee3849c0d035` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.896361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18cb836ebac24ceca14eee3849c0d035)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.898446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:10.908169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa305e30-928c-4210-8262-4132d8be7189 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:10.911072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:10.914771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:10.915930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:10.917097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:10.917461 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] PATH_INFO: `/v3/users/dba218265e1146c786a2b6c6adfb56a1` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:10.919145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dba218265e1146c786a2b6c6adfb56a1)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:10.922934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.002711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18cb836ebac24ceca14eee3849c0d035', 'request_id': 'req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.007332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:11.031401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dba218265e1146c786a2b6c6adfb56a1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.033206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18cb836ebac24ceca14eee3849c0d035', 'request_id': 'req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.057186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18cb836ebac24ceca14eee3849c0d035', 'request_id': 'req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.065421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dba218265e1146c786a2b6c6adfb56a1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.066363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dba218265e1146c786a2b6c6adfb56a1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.082875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] The token cache is being invalidate because project 18cb836ebac24ceca14eee3849c0d035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:11.083439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.084413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b5cfcfad-5d65-4054-8d50-a4d3b20d76f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.086049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dba218265e1146c786a2b6c6adfb56a1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.086995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 947/1914] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:10 2022] DELETE /identity/v3/projects/18cb836ebac24ceca14eee3849c0d035 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:33:11.093770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48c8f-0b93-4046-8e30-ca7bbc232d63 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.094587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-238645d9-a74f-413c-89b5-f44ed643a9b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dba218265e1146c786a2b6c6adfb56a1'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.095133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48c8f-0b93-4046-8e30-ca7bbc232d63 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.095557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab48c8f-0b93-4046-8e30-ca7bbc232d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.130784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 968/1915] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:10 2022] DELETE /identity/v3/users/dba218265e1146c786a2b6c6adfb56a1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:11.135447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.136442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.136875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.147864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1ab48c8f-0b93-4046-8e30-ca7bbc232d63 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:11.161465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1ab48c8f-0b93-4046-8e30-ca7bbc232d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.182048 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:11.184076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.278559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 969/1917] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:11.282012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 948/1917] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:11.289204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.290813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.294752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.295200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.295649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.333599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:11.334806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.347857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 949/1918] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:11.348641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.376402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ea754-1fc6-43e6-873b-6f00a64011f0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.384765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.385422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.385718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.385989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.386622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:11.386923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.388738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.410661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48ab03-3521-4504-8ebe-c5b19b484753 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.418320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.418499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 950/1919] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:11.418584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.418584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.418584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.420014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.430468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.432263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.433776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29391f40-59b0-4a7a-a1c9-7872fc1c426b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.442282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.443202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.445391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.445979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.446594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.448484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.451260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 970/1920] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:33:11.480384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 951/1921] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:33:11.485393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.495370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0834c-d338-4c05-9816-d9a276e2f1a5 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.554504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.555118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.555399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.555675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.556323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.558288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.567681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:11.582183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 971/1922] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:33:11.586114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.597149 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fb03d-784b-4afd-a865-4e55b774b83e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.601828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.602429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.602702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.602971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.603605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.605266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.617418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 952/1923] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:11.622771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.636980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bcd9a0-9572-445c-be41-57af3add6747 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.643545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558617fe-7525-47e0-8ede-70ad81f9a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.644198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558617fe-7525-47e0-8ede-70ad81f9a3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.644431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558617fe-7525-47e0-8ede-70ad81f9a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.644703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558617fe-7525-47e0-8ede-70ad81f9a3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.645374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558617fe-7525-47e0-8ede-70ad81f9a3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.646974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558617fe-7525-47e0-8ede-70ad81f9a3bc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.661405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 972/1924] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:33:11.662952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.673646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd237be-8b4a-4974-8d38-dc6249a0d822 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.678528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.679177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.679484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.679774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] PATH_INFO: `/v3/projects/9f9ef4c430c74eb992cc69a29300bbbf/users/607d592d679041afab6bc55ada5746f3/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.697978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f9ef4c430c74eb992cc69a29300bbbf, user_id=607d592d679041afab6bc55ada5746f3, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.700644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.717155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 953/1925] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:11 2022] PUT /identity/v3/projects/9f9ef4c430c74eb992cc69a29300bbbf/users/607d592d679041afab6bc55ada5746f3/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:11.736168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.736521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.736933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.762796 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.764832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.806636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:11.822681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] MFA Rules not processed for user `607d592d679041afab6bc55ada5746f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:11.825664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.882361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561fbcbd-687a-47f9-a6d2-505bb4c963e8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.888393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:11.889229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:11.889769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:11.890559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] PATH_INFO: `/v3/projects/0d3c098149e044b38565858580112f75` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:11.893078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d3c098149e044b38565858580112f75)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:11.895506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:11.941581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 973/1926] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:11.945957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:11.947556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:11.962803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d3c098149e044b38565858580112f75', 'request_id': 'req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.963583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac54c0c-5f96-44b5-a0e9-84c3c4c30324 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:11.972642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d3c098149e044b38565858580112f75', 'request_id': 'req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.980945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d3c098149e044b38565858580112f75', 'request_id': 'req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.991819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] The token cache is being invalidate because project 0d3c098149e044b38565858580112f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:11.992206 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.992927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e699ea2e-620a-4f39-b5f0-4d00159e4c8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:11.994771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 954/1927] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:11 2022] DELETE /identity/v3/projects/0d3c098149e044b38565858580112f75 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:12.000160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed33b-a8b3-4ca6-bb77-1754891a13a5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.000442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed33b-a8b3-4ca6-bb77-1754891a13a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.000872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aed33b-a8b3-4ca6-bb77-1754891a13a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.029824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.030905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.031189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.031488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] PATH_INFO: `/v3/users/c9c07ccf684743a38f312ec5f465cd85` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.032640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9c07ccf684743a38f312ec5f465cd85)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.034603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.036424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-50aed33b-a8b3-4ca6-bb77-1754891a13a5 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:12.037664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-50aed33b-a8b3-4ca6-bb77-1754891a13a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.046071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 955/1928] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:12.055462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.055743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.056012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.081570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:12.082907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.096087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 956/1929] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:12.098635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.100863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9c07ccf684743a38f312ec5f465cd85'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:12.111327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24326ca7-1c3b-4429-8a50-1c5a5dc192da None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.112625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9c07ccf684743a38f312ec5f465cd85'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:12.113423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9c07ccf684743a38f312ec5f465cd85'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:12.116909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.118001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.118289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.118815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.119452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:12.119749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.121364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.122877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9c07ccf684743a38f312ec5f465cd85'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:12.123673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-b506f0b8-1103-4951-a302-df7ac98cc92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9c07ccf684743a38f312ec5f465cd85'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:12.133211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:12.145351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 957/1930] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:12.146710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.148505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.161337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 974/1931] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:11 2022] DELETE /identity/v3/users/c9c07ccf684743a38f312ec5f465cd85 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:12.163662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.163962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.164243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.164857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71506c58-cd3a-4ae6-b816-ff64980c10ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.187406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:12.188659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.246466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.247412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.247836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.248109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.248916 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.250932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.260618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 975/1932] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:12.272500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.274256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.285468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 958/1933] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:12 2022] POST /identity/v3/projects => generated 373 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:33:12.289716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.299273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:12.341199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bbfca-5bd0-4287-bb40-e9428af9f9cf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.356379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ca588-8480-4d56-8f09-9fbd057390c6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.361142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.361783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.362067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.362345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.362977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.364954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.391638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 959/1934] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:12 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:12.393501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.395934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.396554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.396880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.397163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.404109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955fc95e-ad18-40fa-aa1b-f0e2eff500b9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.407484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.408927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.409462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.410048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.410324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.413526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.413526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.415350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.419809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 976/1935] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:33:12.433352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 960/1936] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:12.438355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.451549 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0fe8e5-cd8e-4ce6-9c9f-bbf1ea8f2224 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.458412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94e6bd2-7e9f-48ef-bca7-55fdae569f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.459100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6bd2-7e9f-48ef-bca7-55fdae569f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.459450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6bd2-7e9f-48ef-bca7-55fdae569f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.459976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94e6bd2-7e9f-48ef-bca7-55fdae569f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.460564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e6bd2-7e9f-48ef-bca7-55fdae569f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.462332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94e6bd2-7e9f-48ef-bca7-55fdae569f7a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.476251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 977/1937] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:12 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:12.480678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.492299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8332d032-8827-4ae4-bf32-2db5ee6c226c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.499900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.500383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.500645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.500913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] PATH_INFO: `/v3/projects/7a03d1b0847548a9bd7992951b8b736f/users/e172a77b57d2422d8254aedfd6544016/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.519502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a03d1b0847548a9bd7992951b8b736f, user_id=e172a77b57d2422d8254aedfd6544016, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.522360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.538572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 961/1938] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:12 2022] PUT /identity/v3/projects/7a03d1b0847548a9bd7992951b8b736f/users/e172a77b57d2422d8254aedfd6544016/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:12.549420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.549781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.550059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.584977 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] MFA Rules not processed for user `e172a77b57d2422d8254aedfd6544016`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:12.586285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.635993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:12.693369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 978/1939] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:12.705739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:12.707490 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:12.771922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134a830a-920c-40b2-b3bf-1c443f83eaef admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:12.775765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4325f66c-0cd8-49a3-b92b-0513d9f767d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:12.777622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325f66c-0cd8-49a3-b92b-0513d9f767d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:12.777622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325f66c-0cd8-49a3-b92b-0513d9f767d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:12.777622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4325f66c-0cd8-49a3-b92b-0513d9f767d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:12.792048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325f66c-0cd8-49a3-b92b-0513d9f767d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:12.793895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4325f66c-0cd8-49a3-b92b-0513d9f767d2 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:12.819379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 962/1940] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:13.000936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.002584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:13.088155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e43467-3ff4-4ece-83ac-e0a0c3526b74 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.101573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.102994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.103269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.103537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] PATH_INFO: `/v3/projects/5b5f57789fe74598914fe21b21991275` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.104693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b5f57789fe74598914fe21b21991275)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.106639 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.165971 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b5f57789fe74598914fe21b21991275', 'request_id': 'req-f5bd4e59-5ddd-435f-908a-160163b33066'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:13.177140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b5f57789fe74598914fe21b21991275', 'request_id': 'req-f5bd4e59-5ddd-435f-908a-160163b33066'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:13.193248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b5f57789fe74598914fe21b21991275', 'request_id': 'req-f5bd4e59-5ddd-435f-908a-160163b33066'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:13.211978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] The token cache is being invalidate because project 5b5f57789fe74598914fe21b21991275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:13.212213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:13.213228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f5bd4e59-5ddd-435f-908a-160163b33066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:13.222185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 979/1941] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:12 2022] DELETE /identity/v3/projects/5b5f57789fe74598914fe21b21991275 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:33:13.225290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.225290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.225290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.261513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:13.268245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:13.356551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 963/1942] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:13.365464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.365751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.366029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.386820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:13.388020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:13.397134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 980/1943] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:13.401299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.411176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b55b1ec-8552-4a6c-b344-5fea1b9a0b9f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.415877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.416501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.416780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.417114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.417799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:13.418102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.419782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.421779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:13.433561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 964/1944] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:13.438494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.447835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dd82a-e723-44e9-a261-07f9d2433524 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.453110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.453848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.454128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.454399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.455037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.456941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.483453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 981/1945] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:13 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:33:13.488054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.498370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb341b6-2f36-4c4f-a66c-9bcdb66bdbaa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.552703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.553330 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.553609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.553876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.554484 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.556388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.579254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 965/1946] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:13 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:13.583184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.595296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c4ef6a-b832-4e8b-9b65-ac327e2b287b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.600632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.601368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.601637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.601905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.602546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.604146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.615552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 982/1947] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:13.619635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.630269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3402787-0b4b-411f-8949-402875a49493 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.637273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866118ad-8c20-480d-9078-ef96a5d312b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.638334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866118ad-8c20-480d-9078-ef96a5d312b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.638821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866118ad-8c20-480d-9078-ef96a5d312b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.639315 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866118ad-8c20-480d-9078-ef96a5d312b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.641082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866118ad-8c20-480d-9078-ef96a5d312b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.643087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866118ad-8c20-480d-9078-ef96a5d312b0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.653907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 966/1948] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:13.658276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.668478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5747a4db-1475-474a-b009-0564de408a7c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.673385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.674029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.674304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.674571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] PATH_INFO: `/v3/projects/cd6efe6c98a9432b8fcf69ec4e4dfb2d/users/f026ca87a01a4870860e603f28d5a869/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.690755 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd6efe6c98a9432b8fcf69ec4e4dfb2d, user_id=f026ca87a01a4870860e603f28d5a869, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.693496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.695934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:13.709229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 983/1949] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:13 2022] PUT /identity/v3/projects/cd6efe6c98a9432b8fcf69ec4e4dfb2d/users/f026ca87a01a4870860e603f28d5a869/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:13.720457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.720768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.721061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.748220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] MFA Rules not processed for user `f026ca87a01a4870860e603f28d5a869`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:13.750173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:13.817443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 967/1950] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:13.828598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:13.830316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:13.892809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc795b5d-3b6b-43f5-af42-8b5f9c8db887 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:13.898787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946405c-334a-4fbe-88d8-a3c9c64bd76d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:13.898787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946405c-334a-4fbe-88d8-a3c9c64bd76d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:13.898787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946405c-334a-4fbe-88d8-a3c9c64bd76d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:13.898787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946405c-334a-4fbe-88d8-a3c9c64bd76d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:13.906688 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946405c-334a-4fbe-88d8-a3c9c64bd76d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:13.908125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7946405c-334a-4fbe-88d8-a3c9c64bd76d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:13.916953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 984/1951] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:17.571351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:17.637761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1308-da47-4b47-b976-66bc3a543b5c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:17.643259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:17.643902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:17.644177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:17.644450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] PATH_INFO: `/v3/users/f026ca87a01a4870860e603f28d5a869` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:17.645604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f026ca87a01a4870860e603f28d5a869)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:17.648226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:17.652402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:17.704160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f026ca87a01a4870860e603f28d5a869'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:17.704836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f026ca87a01a4870860e603f28d5a869'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:17.705470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f026ca87a01a4870860e603f28d5a869'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:17.713940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f026ca87a01a4870860e603f28d5a869'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:17.727946 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f026ca87a01a4870860e603f28d5a869'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:17.750587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 968/1952] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:17 2022] DELETE /identity/v3/users/f026ca87a01a4870860e603f28d5a869 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:17.755041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:17.755485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:17.755576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:17.775042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:17.776234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:17.837555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 985/1953] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:17.848048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:17.849648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:17.908352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c6ddb-24ad-40c7-8561-a8d36da7a805 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:17.911991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65180a28-f24c-4ee8-9e0b-9b99ddd51ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:17.912592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65180a28-f24c-4ee8-9e0b-9b99ddd51ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:17.912866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65180a28-f24c-4ee8-9e0b-9b99ddd51ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:17.913132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65180a28-f24c-4ee8-9e0b-9b99ddd51ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:17.923027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65180a28-f24c-4ee8-9e0b-9b99ddd51ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:17.924493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65180a28-f24c-4ee8-9e0b-9b99ddd51ea1 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:17.933889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 969/1954] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:18.228944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.230733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:18.245990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b7651-d4f7-45d9-8b61-24c2ad33f35e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.251561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.252503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.253040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.253351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] PATH_INFO: `/v3/projects/cd6efe6c98a9432b8fcf69ec4e4dfb2d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.254563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd6efe6c98a9432b8fcf69ec4e4dfb2d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.256402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.314078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd6efe6c98a9432b8fcf69ec4e4dfb2d', 'request_id': 'req-af905253-ebab-499a-8f6c-7824e7781f6e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:18.320949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd6efe6c98a9432b8fcf69ec4e4dfb2d', 'request_id': 'req-af905253-ebab-499a-8f6c-7824e7781f6e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:18.330061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd6efe6c98a9432b8fcf69ec4e4dfb2d', 'request_id': 'req-af905253-ebab-499a-8f6c-7824e7781f6e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:18.340571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] The token cache is being invalidate because project cd6efe6c98a9432b8fcf69ec4e4dfb2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:18.340853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:18.341518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-af905253-ebab-499a-8f6c-7824e7781f6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:18.343054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 986/1955] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:18 2022] DELETE /identity/v3/projects/cd6efe6c98a9432b8fcf69ec4e4dfb2d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:18.347906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.348278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.348581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.369433 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:18.370950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:18.438261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 970/1956] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:18.446175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.446457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.446735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.467662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:18.468958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:18.479525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 987/1957] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:18.484711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.494857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f585d17-1e67-4362-ab69-70f746aa893b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.500491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.501083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.501416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.501702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.504474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:18.504474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.504474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.516708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 971/1958] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:18.521078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.530219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b3317b-920a-46bf-9426-9584bda0780a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.535335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.535866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.536148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.536420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.537184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.539135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.554788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:18.580394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 988/1959] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:18 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:18.584470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.598857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3a5b0c-578b-4025-a5ff-b63ec68bf6e6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.607387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:18.666169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.666885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.667050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.667323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.667950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.669905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.697348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 972/1960] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:18 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:18.700084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.710270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb174a0-c220-4072-9421-562d90da74c8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.714893 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.715490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.715767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.716038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.716671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.718894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.729451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 989/1961] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:18.734911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.755372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd7bdfc-ca29-4863-bba0-8ac6ba3f092d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.760101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ea525d-b2c1-4f7a-b54b-1edaa944e2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.760718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea525d-b2c1-4f7a-b54b-1edaa944e2cc admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.761007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea525d-b2c1-4f7a-b54b-1edaa944e2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.761275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ea525d-b2c1-4f7a-b54b-1edaa944e2cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.762089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea525d-b2c1-4f7a-b54b-1edaa944e2cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.763934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ea525d-b2c1-4f7a-b54b-1edaa944e2cc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.776845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 973/1962] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:18.781141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:18.795239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855635d-731b-453a-9609-bd3e6308aab3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:18.800884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:18.801577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.801859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.802135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] PATH_INFO: `/v3/projects/e623a3cfd7224c7c9815b61cb7f8d10c/users/e9c566823d3e4f6abeb4454c12ab0fff/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.819083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e623a3cfd7224c7c9815b61cb7f8d10c, user_id=e9c566823d3e4f6abeb4454c12ab0fff, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:18.821713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:18.838160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 990/1963] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:18 2022] PUT /identity/v3/projects/e623a3cfd7224c7c9815b61cb7f8d10c/users/e9c566823d3e4f6abeb4454c12ab0fff/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:18.858056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:18.858240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:18.858834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:18.902524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] MFA Rules not processed for user `e9c566823d3e4f6abeb4454c12ab0fff`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:18.903888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:18.994650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 974/1964] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:19.007213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:19.008974 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:19.083163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59af3592-16a7-41a4-aff6-cd31719b2b8a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:19.090817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:19.091559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:19.091913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:19.092329 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:19.102575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:19.104166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:19.117079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 991/1965] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:19 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:33:25.639789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:25.641856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:25.728855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60309b67-ade3-45e1-ab72-854e4ee8cd64 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:25.732563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:25.733802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:25.733802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:25.733802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:25.736253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:25.751696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:25.798097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:25.799559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e763a1-891b-4891-b8a9-dd0ffe7b122a service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:25.808392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 975/1966] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:29.110945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:35.804380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=21 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:53.804837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:53.806287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:53.874036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e383e7b-f810-4c69-8199-a63dc05dbced service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:53.879245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:53.879891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:53.880175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:53.880448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] PATH_INFO: `/v3/users/e172a77b57d2422d8254aedfd6544016` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:53.881647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e172a77b57d2422d8254aedfd6544016)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:53.883587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:53.930586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e172a77b57d2422d8254aedfd6544016'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:53.937228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e172a77b57d2422d8254aedfd6544016'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:53.937868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e172a77b57d2422d8254aedfd6544016'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:53.945419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e172a77b57d2422d8254aedfd6544016'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:53.946045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e172a77b57d2422d8254aedfd6544016'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:53.964467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 992/1967] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:53 2022] DELETE /identity/v3/users/e172a77b57d2422d8254aedfd6544016 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:53.969676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:53.970199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:53.970646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:53.997561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:53.998794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:54.060411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 976/1968] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:54.070842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.072245 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:54.129819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232948d4-f29d-4c83-b73b-3d2b3b81ad84 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.133435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99680285-ecce-478c-85f6-bcd3ee243ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.134037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99680285-ecce-478c-85f6-bcd3ee243ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.134313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99680285-ecce-478c-85f6-bcd3ee243ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.134580 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99680285-ecce-478c-85f6-bcd3ee243ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.144057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99680285-ecce-478c-85f6-bcd3ee243ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.145491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99680285-ecce-478c-85f6-bcd3ee243ba6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.154729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 993/1969] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:54.441242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.445800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:54.461489 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b4a9c-7920-4283-9d51-31383a37a57f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.467293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.468271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.468702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.468996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] PATH_INFO: `/v3/projects/7a03d1b0847548a9bd7992951b8b736f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.470715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a03d1b0847548a9bd7992951b8b736f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.473632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.529085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a03d1b0847548a9bd7992951b8b736f', 'request_id': 'req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:54.537815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a03d1b0847548a9bd7992951b8b736f', 'request_id': 'req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:54.544674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a03d1b0847548a9bd7992951b8b736f', 'request_id': 'req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:54.556135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] The token cache is being invalidate because project 7a03d1b0847548a9bd7992951b8b736f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:54.556505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:54.557170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-9f47356f-a6c1-4f66-8b2f-e8c4d58d11a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:54.559002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 977/1970] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:54 2022] DELETE /identity/v3/projects/7a03d1b0847548a9bd7992951b8b736f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:54.565503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.565902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.566243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.586489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:54.587686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:54.606504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:54.654133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 994/1971] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:54.667542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.667764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.668090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.695998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:54.697465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:54.706659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 978/1972] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:54.710709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.720996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf66868-a17b-4226-aed0-ab51b822fda2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.725788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.726380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.726673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.726945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.727599 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:54.727898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.729552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.741726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 995/1973] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:54.746377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.757091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f9ac6-3fdb-4d30-8096-937805c893ba None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.762191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.762739 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.763019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.763323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.764071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.766057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.796085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 979/1974] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:54 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:54.800626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.812941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567cf8d-839b-44ae-9167-32c4f1f5a932 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.879096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.879708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.880019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.880294 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.880938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.882902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.904664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 996/1975] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:54 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:33:54.908913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.919809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9263ad-d528-4567-86b8-3ed308334aeb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.925378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.925684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.925991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.926265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.926912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.928519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.938896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 980/1976] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:54.943743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.954165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a934c1-cd8a-45aa-9b25-9c561463e41f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.959096 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401a9f4-5f12-4a6a-b4b3-19825b158fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.959717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401a9f4-5f12-4a6a-b4b3-19825b158fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.959999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401a9f4-5f12-4a6a-b4b3-19825b158fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.960272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401a9f4-5f12-4a6a-b4b3-19825b158fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:54.960944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401a9f4-5f12-4a6a-b4b3-19825b158fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:54.962581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401a9f4-5f12-4a6a-b4b3-19825b158fd2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:54.973085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 997/1977] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:54.977252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:54.987809 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6970b6-32da-4c35-b19c-285f5fe209f4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:54.992401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:54.993046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:54.993351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:54.993633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] PATH_INFO: `/v3/projects/ded843c641e84227bfadd4d9d590317c/users/d1fbfe1a7b4347a6a28241a8e8d815e0/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:55.008903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ded843c641e84227bfadd4d9d590317c, user_id=d1fbfe1a7b4347a6a28241a8e8d815e0, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:55.011656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:55.026703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 981/1978] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:54 2022] PUT /identity/v3/projects/ded843c641e84227bfadd4d9d590317c/users/d1fbfe1a7b4347a6a28241a8e8d815e0/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:55.032034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:55.032488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:55.033006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:55.053880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:55.055239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:55.119278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 998/1979] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:55.131302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:55.133064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:55.154669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:55.194413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4663af-057b-4962-901b-4bbe79728620 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:55.198038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593c697-3ced-4e1e-a8c8-204c1509f461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:55.198641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c697-3ced-4e1e-a8c8-204c1509f461 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:55.198941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c697-3ced-4e1e-a8c8-204c1509f461 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:55.199210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593c697-3ced-4e1e-a8c8-204c1509f461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:55.208515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593c697-3ced-4e1e-a8c8-204c1509f461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:55.210684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593c697-3ced-4e1e-a8c8-204c1509f461 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:55.220670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 982/1980] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:33:58.217122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:58.217824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:58.239940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b6ba3b-be91-423f-963f-a7d4f87893b1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:58.248988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:58.249686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:58.249970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:58.250243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] PATH_INFO: `/v3/users/e9c566823d3e4f6abeb4454c12ab0fff` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:58.252235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9c566823d3e4f6abeb4454c12ab0fff)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:58.254919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:58.301578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:58.335053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9c566823d3e4f6abeb4454c12ab0fff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:58.345603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9c566823d3e4f6abeb4454c12ab0fff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:58.346925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9c566823d3e4f6abeb4454c12ab0fff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:58.355069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9c566823d3e4f6abeb4454c12ab0fff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:58.355729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9c566823d3e4f6abeb4454c12ab0fff'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:58.382858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 999/1981] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:33:58 2022] DELETE /identity/v3/users/e9c566823d3e4f6abeb4454c12ab0fff => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:58.386129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:58.388392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:58.389338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:58.426113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:58.427335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:58.503179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 983/1982] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:58.509962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:58.511702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:58.613371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5d031-3ae7-40f8-a521-ba85b8dec298 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:58.618949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a4815-c454-45a1-925a-4dd686b7d6e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:58.619986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a4815-c454-45a1-925a-4dd686b7d6e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:58.620660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a4815-c454-45a1-925a-4dd686b7d6e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:58.621089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a4815-c454-45a1-925a-4dd686b7d6e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:58.631900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a4815-c454-45a1-925a-4dd686b7d6e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:58.633959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a4815-c454-45a1-925a-4dd686b7d6e6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:58.647895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1000/1983] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:33:59.040715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.042215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:59.065454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b558277-3dcd-4dae-91c3-8053eeb4c58c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.071207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.072181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.073050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.073050 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] PATH_INFO: `/v3/projects/e623a3cfd7224c7c9815b61cb7f8d10c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.075234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e623a3cfd7224c7c9815b61cb7f8d10c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.078742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.145669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e623a3cfd7224c7c9815b61cb7f8d10c', 'request_id': 'req-7e342380-d19d-4b66-b097-336b9cceacb3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:59.156631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e623a3cfd7224c7c9815b61cb7f8d10c', 'request_id': 'req-7e342380-d19d-4b66-b097-336b9cceacb3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:59.165168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e623a3cfd7224c7c9815b61cb7f8d10c', 'request_id': 'req-7e342380-d19d-4b66-b097-336b9cceacb3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:59.177544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] The token cache is being invalidate because project e623a3cfd7224c7c9815b61cb7f8d10c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:33:59.178529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:59.179262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7e342380-d19d-4b66-b097-336b9cceacb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:33:59.184447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 984/1984] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:33:59 2022] DELETE /identity/v3/projects/e623a3cfd7224c7c9815b61cb7f8d10c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:59.186013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.186300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.186572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.207910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:59.209133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:59.290823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1001/1985] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:59.298880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.299790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.300109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.327925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:59.329611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:59.339447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 985/1986] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:33:59.343295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.356799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b5fab-0ba0-4077-ad30-1985d819d1e7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.371847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.373583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.373888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.374152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.376029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:33:59.376029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.379382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.400526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1002/1987] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:33:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:33:59.404366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.425359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587ef32-5747-4947-9cc6-aecdc15e3a5f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.443282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.443912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.444189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.444590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.445240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.447316 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.489996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 986/1988] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/projects => generated 379 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:59.494358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.511728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f39acb4-66a6-465e-b71b-cb83c5e9917f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.619257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:59.693832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acc547d-8f7b-4818-b6c9-31d477893e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.694768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc547d-8f7b-4818-b6c9-31d477893e8c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.698886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc547d-8f7b-4818-b6c9-31d477893e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.699192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acc547d-8f7b-4818-b6c9-31d477893e8c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.699997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc547d-8f7b-4818-b6c9-31d477893e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.701968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acc547d-8f7b-4818-b6c9-31d477893e8c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.736459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1003/1989] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:33:59.738109 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.750928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e663e-86e7-4b69-97f8-c0a79587becb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.764492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.764994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.765469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.773653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.775082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.775358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.775627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.776293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.777992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.794332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 987/1990] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:59.797656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.812396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] MFA Rules not processed for user `d1fbfe1a7b4347a6a28241a8e8d815e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:33:59.813691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:59.819481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f9018-b0a0-44fb-afdb-407b71efb7cb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.828186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.829237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.829772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.830364 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.832233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.834683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.837262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:33:59.864913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 988/1991] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:33:59 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:33:59.869996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.893179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18f039-8ecc-4aa6-8cfd-55600ac07f02 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.902048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678c686c-1d2f-4d64-a709-867162b7d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:33:59.902988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c686c-1d2f-4d64-a709-867162b7d9ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.903414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c686c-1d2f-4d64-a709-867162b7d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.903855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678c686c-1d2f-4d64-a709-867162b7d9ea admin admin] PATH_INFO: `/v3/projects/9e1a9bafab47407599e09fb4d0f8262b/users/4b702145173b471e880425c0703b4d7c/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:33:59.925042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c686c-1d2f-4d64-a709-867162b7d9ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e1a9bafab47407599e09fb4d0f8262b, user_id=4b702145173b471e880425c0703b4d7c, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:33:59.931566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678c686c-1d2f-4d64-a709-867162b7d9ea admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:33:59.953064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1004/1992] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:33:59.955452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:33:59.957080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:33:59.968451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 989/1993] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:33:59 2022] PUT /identity/v3/projects/9e1a9bafab47407599e09fb4d0f8262b/users/4b702145173b471e880425c0703b4d7c/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:33:59.974261 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:33:59.974529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545fc8f-3deb-4d6c-ad21-b0ee41d7b2e0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:33:59.974643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:33:59.974823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.000450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:00.002550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:00.066774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.067402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.067716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.067948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.068590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.070592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.087273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 990/1994] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:00.098098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:00.100423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:00.114635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1005/1995] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:33:59 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:00.115696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:00.170117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:00.171868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f6ec5-3778-460f-bdc9-67039d733938 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:00.212118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a46e4-76ef-4c9c-a527-d1981aeca9cb admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:00.220531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.221940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.222228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.222492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.223125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.225276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.242440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.243101 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.243510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.243922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.250429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1006/1996] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:00 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:34:00.254164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:00.255705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.257154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.267566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02366bd6-351c-47c7-a84f-01a15a10d69b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:00.273509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.273848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 991/1997] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:34:00.274445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.274731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.275009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.277867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.279990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.292993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1007/1998] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:34:00.297293 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:00.308650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25db52-b251-4f3a-9945-781c4f6d15d0 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:00.314854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.315417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.315760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.316084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] PATH_INFO: `/v3/projects/14383ebc601b4a16aab303b4c24925f5/users/0ff542c9edb647019d9b38eee76e68bf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.339882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14383ebc601b4a16aab303b4c24925f5, user_id=0ff542c9edb647019d9b38eee76e68bf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.342879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.360131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 992/1999] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:00 2022] PUT /identity/v3/projects/14383ebc601b4a16aab303b4c24925f5/users/0ff542c9edb647019d9b38eee76e68bf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:00.367125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:00.382126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c80d-09a4-478f-80b9-97e38e68b292 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:00.503059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e0628-e0e7-466a-ad20-830ae1bc54d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.504358 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e0628-e0e7-466a-ad20-830ae1bc54d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.504738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e0628-e0e7-466a-ad20-830ae1bc54d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.505672 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e0628-e0e7-466a-ad20-830ae1bc54d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.506420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e0628-e0e7-466a-ad20-830ae1bc54d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.508068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e0628-e0e7-466a-ad20-830ae1bc54d7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.521608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1008/2000] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:00.524908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:00.545204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3584d2f-138c-4af6-a09d-005903ea6f51 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:00.554896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:00.555587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:00.556030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:00.556449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] PATH_INFO: `/v3/projects/14383ebc601b4a16aab303b4c24925f5/users/0ff542c9edb647019d9b38eee76e68bf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:00.559044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14383ebc601b4a16aab303b4c24925f5, user_id=0ff542c9edb647019d9b38eee76e68bf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:00.562005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:00.565296 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:00.576730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 993/2001] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:00 2022] PUT /identity/v3/projects/14383ebc601b4a16aab303b4c24925f5/users/0ff542c9edb647019d9b38eee76e68bf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:08.230333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70b8f4-d23a-492b-b226-956f07d5f249 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:08.230870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70b8f4-d23a-492b-b226-956f07d5f249 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:08.230870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b70b8f4-d23a-492b-b226-956f07d5f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:08.309345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0b70b8f4-d23a-492b-b226-956f07d5f249 None None] MFA Rules not processed for user `0ff542c9edb647019d9b38eee76e68bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:08.313436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0b70b8f4-d23a-492b-b226-956f07d5f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:08.421354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1009/2002] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:08.437473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:08.440509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:08.577370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73080d-4140-4e9b-9614-d376ead1a578 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:08.584071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28298997-abb6-4457-a295-711726da4085 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:08.585153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28298997-abb6-4457-a295-711726da4085 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:08.585791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28298997-abb6-4457-a295-711726da4085 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:08.586430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28298997-abb6-4457-a295-711726da4085 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:08.687696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e722f0-1aa0-4a12-b2fd-84869e553c41 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:08.687990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e722f0-1aa0-4a12-b2fd-84869e553c41 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:08.688270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e722f0-1aa0-4a12-b2fd-84869e553c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:08.714453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28298997-abb6-4457-a295-711726da4085 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:08.715923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28298997-abb6-4457-a295-711726da4085 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:08.733463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 994/2003] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:08.734155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e6e722f0-1aa0-4a12-b2fd-84869e553c41 None None] MFA Rules not processed for user `4b702145173b471e880425c0703b4d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:08.736090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e6e722f0-1aa0-4a12-b2fd-84869e553c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:08.790793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:08.879679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1010/2004] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:34:08.884188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28298997-abb6-4457-a295-711726da4085 service glance] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:08.936862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:08.937288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:08.938165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:08.967465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:08.968851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:08.971360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28298997-abb6-4457-a295-711726da4085 service glance] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:08.976122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:08.976753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:08.977036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:08.977342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:08.978135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:08.980767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.014807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 995/2005] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:34:08 2022] POST /identity/v3/projects => generated 377 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:09.021649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.038560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1011/2006] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:34:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:09.071334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:09.097495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.098996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:09.125492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f440997-bd38-470f-b952-f9d70f584f4b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.131950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.132830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.133491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.133623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.134385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.137083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.187032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 996/2007] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:09 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:09.188497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.208973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437c0811-adce-4378-a906-dab9773c5de6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.220001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c57bd-9129-47e1-a995-67081aea16f5 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.224844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.230345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.231362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.233003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.233803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.238872 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.257256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 997/2008] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:09.258923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.268519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:09.350724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.352404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.352404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.352404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.369564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:09.389747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd514caf-df1f-4ac3-a228-449cbd5e0195 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.400713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.401878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.402305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.402827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.407147 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:09.425599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.427588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.450641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 998/2009] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:09.452354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.452500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.454157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.467044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1012/2010] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:34:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:34:09.467442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b0146d-bbf2-4300-b736-8455a3c18050 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.473256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.474093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.474288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.474568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] PATH_INFO: `/v3/projects/19d39455ff9d4c8680f344fd18fb072d/users/055839f062d644059a9cf7925cf7f16b/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.496482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d39455ff9d4c8680f344fd18fb072d, user_id=055839f062d644059a9cf7925cf7f16b, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.500735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.574581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 999/2011] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:09 2022] PUT /identity/v3/projects/19d39455ff9d4c8680f344fd18fb072d/users/055839f062d644059a9cf7925cf7f16b/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:09.578661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.598280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625e3c1-9c1f-470e-8d89-3e68eca388b3 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.789300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b29857-b6e1-466c-9f3e-7e24e21cba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.790011 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b29857-b6e1-466c-9f3e-7e24e21cba55 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.790783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b29857-b6e1-466c-9f3e-7e24e21cba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.790783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b29857-b6e1-466c-9f3e-7e24e21cba55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.791196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b29857-b6e1-466c-9f3e-7e24e21cba55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.793624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b29857-b6e1-466c-9f3e-7e24e21cba55 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.822350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1013/2012] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:09.826520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:09.846812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dc9235-b3d1-458c-874a-2031b653798b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:09.852066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:09.852993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:09.853274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:09.853659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] PATH_INFO: `/v3/projects/19d39455ff9d4c8680f344fd18fb072d/users/055839f062d644059a9cf7925cf7f16b/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:09.855460 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19d39455ff9d4c8680f344fd18fb072d, user_id=055839f062d644059a9cf7925cf7f16b, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:09.858176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:09.885646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1000/2013] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:09 2022] PUT /identity/v3/projects/19d39455ff9d4c8680f344fd18fb072d/users/055839f062d644059a9cf7925cf7f16b/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:17.044810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa6b45-3bbb-4d63-8e50-8234ff277eb1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:17.045864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa6b45-3bbb-4d63-8e50-8234ff277eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:17.046501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa6b45-3bbb-4d63-8e50-8234ff277eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:17.099695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-25fa6b45-3bbb-4d63-8e50-8234ff277eb1 None None] MFA Rules not processed for user `76e3059b01884215becf97694c3b7027`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:17.101213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-25fa6b45-3bbb-4d63-8e50-8234ff277eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:17.287575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1014/2014] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:17 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:18.113023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:18.115606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:18.293943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cc33c7-ad6b-4cf9-945d-582822f6a569 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:18.300916 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:18.301790 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:18.302231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:18.302558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] PATH_INFO: `/v3/users/63b310e542594e3498bc5a0198d53b9f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:18.303841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b310e542594e3498bc5a0198d53b9f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:18.307195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:18.313801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:18.417359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b310e542594e3498bc5a0198d53b9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:18.418063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b310e542594e3498bc5a0198d53b9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:18.418690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b310e542594e3498bc5a0198d53b9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:18.434388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b310e542594e3498bc5a0198d53b9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:18.446956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b310e542594e3498bc5a0198d53b9f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:18.476663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1001/2015] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:34:18 2022] DELETE /identity/v3/users/63b310e542594e3498bc5a0198d53b9f => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:18.480966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f9e5f-c8cf-4d1f-b710-768469267b7d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:18.481442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f9e5f-c8cf-4d1f-b710-768469267b7d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:18.481744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f9e5f-c8cf-4d1f-b710-768469267b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:18.524662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a5f9e5f-c8cf-4d1f-b710-768469267b7d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:18.525900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a5f9e5f-c8cf-4d1f-b710-768469267b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:18.632859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:18.679810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1015/2016] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:18.689035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:18.691033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:18.843833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59562c2c-21b6-4bb9-bd68-1fefa96e9289 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:18.856602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:18.857721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:18.858191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:18.858634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:18.887582 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:18.889054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:18.918992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1002/2017] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:19.434222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:19.434508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:19.434785 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:19.538870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] MFA Rules not processed for user `055839f062d644059a9cf7925cf7f16b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:19.540431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:19.544090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:19.545891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:19.562935 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7662ba9-1907-4e75-8941-1069e89071c1 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:19.582721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:19.584111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:19.584606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:19.585118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] PATH_INFO: `/v3/projects/b99c469c3ab149819915024f478d80a1` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:19.586590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b99c469c3ab149819915024f478d80a1)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:19.588972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:19.659242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1016/2018] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:19.678508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:19.686933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:19.695049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b99c469c3ab149819915024f478d80a1', 'request_id': 'req-b1bf8506-df0f-4468-a10b-7cf1e371bed5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:19.724165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b99c469c3ab149819915024f478d80a1', 'request_id': 'req-b1bf8506-df0f-4468-a10b-7cf1e371bed5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:19.735942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b99c469c3ab149819915024f478d80a1', 'request_id': 'req-b1bf8506-df0f-4468-a10b-7cf1e371bed5'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:19.751992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] The token cache is being invalidate because project b99c469c3ab149819915024f478d80a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:34:19.753036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:19.753738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b1bf8506-df0f-4468-a10b-7cf1e371bed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:19.756129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1003/2019] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:34:19 2022] DELETE /identity/v3/projects/b99c469c3ab149819915024f478d80a1 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:34:19.760723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008c1b-2683-4fae-bd85-b9e345c0e797 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:19.761085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008c1b-2683-4fae-bd85-b9e345c0e797 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:19.761480 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3008c1b-2683-4fae-bd85-b9e345c0e797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:19.798922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f3008c1b-2683-4fae-bd85-b9e345c0e797 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:19.801047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f3008c1b-2683-4fae-bd85-b9e345c0e797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:19.806651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:19.904049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f4150e-974b-4a97-a787-ac1a88f6adbf None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:19.939952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1004/2020] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:34:19.963463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:19.963463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:19.963463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.013372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:20.014831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:20.034966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1005/2021] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:20.039992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.053532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.053944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f23418a-c580-416e-87f5-a888e8a08181 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:20.054541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.054978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.055403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.058269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:20.074042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.074776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.075134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.075474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.076036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:20.077666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:34:20.078115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.082332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.105064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1006/2022] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:34:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:34:20.109930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.123578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19177a27-f930-490d-a04c-4235e2df381f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:20.137133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.137975 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.138417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.139435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.143766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.146513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.168546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.170485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.183061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1017/2023] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:20.227627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1007/2024] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:34:20 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:20.231657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.257075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f31a24-4528-4419-9ea1-12325e9c6d4d service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:20.426316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.427095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.427476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.427858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.428596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.431762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.482151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1018/2025] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:20 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:20.485934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.514268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88d50e6-8fe6-4456-b2aa-daa0e766f02a admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:20.543322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.544329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.546003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.546547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.547428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.550242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.569824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1008/2026] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:34:20.574702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.589508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c1e8c3-ae6f-42f3-ba2c-60f5b40b90d8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:20.602249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0902724-6148-44d7-8644-855b32968493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.602913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0902724-6148-44d7-8644-855b32968493 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.603145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0902724-6148-44d7-8644-855b32968493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.603590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0902724-6148-44d7-8644-855b32968493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.604242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0902724-6148-44d7-8644-855b32968493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.606353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0902724-6148-44d7-8644-855b32968493 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.620878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1019/2027] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:20.623655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.652533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52740dfd-a505-4d79-8e4b-100580fb7b5c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:20.664927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:20.668767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.668767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.668767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] PATH_INFO: `/v3/projects/af1b620b3ebe475b8a6d82e4cb5d5097/users/298ebaa6478544dba2df00dabb44ad46/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.694727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] RBAC: Authorizing `identity:create_grant(project_id=af1b620b3ebe475b8a6d82e4cb5d5097, user_id=298ebaa6478544dba2df00dabb44ad46, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:20.697651 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:20.721358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1009/2028] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:20 2022] PUT /identity/v3/projects/af1b620b3ebe475b8a6d82e4cb5d5097/users/298ebaa6478544dba2df00dabb44ad46/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:20.733083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebe03-a389-4df8-b03a-2fbe70390793 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:20.734149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebe03-a389-4df8-b03a-2fbe70390793 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:20.734734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9ebe03-a389-4df8-b03a-2fbe70390793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:20.782757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9c9ebe03-a389-4df8-b03a-2fbe70390793 None None] MFA Rules not processed for user `298ebaa6478544dba2df00dabb44ad46`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:20.784530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9c9ebe03-a389-4df8-b03a-2fbe70390793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:20.912770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1020/2029] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:20.929906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:20.933504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:21.074570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a252b4-11d7-4758-9a66-a3d07dae767e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:21.079079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ac4e9-24b7-474d-8838-1f61cb8e6145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:21.080028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ac4e9-24b7-474d-8838-1f61cb8e6145 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:21.081214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ac4e9-24b7-474d-8838-1f61cb8e6145 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:21.081731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ac4e9-24b7-474d-8838-1f61cb8e6145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:21.093855 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ac4e9-24b7-474d-8838-1f61cb8e6145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:21.095889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ac4e9-24b7-474d-8838-1f61cb8e6145 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:21.107274 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1010/2030] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:23.996572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:23.997193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:23.997193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:24.031030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] MFA Rules not processed for user `298ebaa6478544dba2df00dabb44ad46`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:24.032540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:24.042194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1021/2031] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:27.003411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15279062-4c71-4ccb-a5fb-08d723e4ce44 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:27.004004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15279062-4c71-4ccb-a5fb-08d723e4ce44 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:27.004004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15279062-4c71-4ccb-a5fb-08d723e4ce44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:27.022607 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-15279062-4c71-4ccb-a5fb-08d723e4ce44 None None] MFA Rules not processed for user `d1fbfe1a7b4347a6a28241a8e8d815e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:27.023804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15279062-4c71-4ccb-a5fb-08d723e4ce44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:27.055211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:27.088743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1011/2032] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:29.599290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:29.600749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:29.626731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:29.685336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee354c-3dea-4d64-8123-718c999bb437 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:29.690088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:29.690722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:29.690997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:29.691267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] PATH_INFO: `/v3/users/607d592d679041afab6bc55ada5746f3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:29.692422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=607d592d679041afab6bc55ada5746f3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:29.694552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:29.742795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '607d592d679041afab6bc55ada5746f3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:29.750195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '607d592d679041afab6bc55ada5746f3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:29.750808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '607d592d679041afab6bc55ada5746f3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:29.757788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '607d592d679041afab6bc55ada5746f3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:29.758429 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '607d592d679041afab6bc55ada5746f3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:29.776306 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1022/2033] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:34:29 2022] DELETE /identity/v3/users/607d592d679041afab6bc55ada5746f3 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:34:29.780503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:29.780972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:29.781094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:29.802255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:29.803488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:29.871157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1012/2034] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:29.883190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:29.884658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:29.957196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a032373-55cd-4882-b722-0726df43be0e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:29.961114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e4f0df-283d-4753-a52c-d90a322eeae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:29.961757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4f0df-283d-4753-a52c-d90a322eeae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:29.962037 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4f0df-283d-4753-a52c-d90a322eeae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:29.962307 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e4f0df-283d-4753-a52c-d90a322eeae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:29.973606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4f0df-283d-4753-a52c-d90a322eeae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:29.975109 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e4f0df-283d-4753-a52c-d90a322eeae0 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:29.993005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1023/2035] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:30.338423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.339884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.354706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f9f73-84b0-492b-9a9d-dd20938def85 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.360537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.361179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.361539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.361971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] PATH_INFO: `/v3/projects/9f9ef4c430c74eb992cc69a29300bbbf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.363258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f9ef4c430c74eb992cc69a29300bbbf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.365185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.421903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f9ef4c430c74eb992cc69a29300bbbf', 'request_id': 'req-5ca3661d-9405-41ea-9564-402b1b4f9df8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:30.432594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f9ef4c430c74eb992cc69a29300bbbf', 'request_id': 'req-5ca3661d-9405-41ea-9564-402b1b4f9df8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:30.440475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f9ef4c430c74eb992cc69a29300bbbf', 'request_id': 'req-5ca3661d-9405-41ea-9564-402b1b4f9df8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:30.451833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] The token cache is being invalidate because project 9f9ef4c430c74eb992cc69a29300bbbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:34:30.453384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:30.454124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5ca3661d-9405-41ea-9564-402b1b4f9df8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:30.455962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1013/2036] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:34:30 2022] DELETE /identity/v3/projects/9f9ef4c430c74eb992cc69a29300bbbf => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:30.460648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.461183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.461551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.483251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:30.484490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.520072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:30.548778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1024/2037] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:30.558259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.558673 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.559162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.580987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:30.582245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:30.590192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1014/2038] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:30.594487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.604823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c72c5a-7f95-48a2-9176-a41bd992a5c7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.609665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.610277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.610559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.610832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.612378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:34:30.612731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.614384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.626905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1025/2039] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:34:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:34:30.632099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.641834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa964a7-d5d1-43a5-b45f-b3644b04d0b7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.647668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.648517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.648869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.649153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.649899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.651857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.682887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1015/2040] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:30.684241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.695192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bdb29-c316-46a9-b497-c0f2bbcb8684 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.755719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.756308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.756585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.756849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.757503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.759453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.787747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1026/2041] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:30.789010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.799444 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b72a0f-dd8a-4c1d-9356-49e9b6535178 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.804345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.804941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.805217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.805814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.806641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.808488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.820211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1016/2042] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:34:30.824145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.834664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca889f5-804d-4c08-a027-5b47d904305f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.839155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a6a79-f0ea-4dbd-af97-e53707fae6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.839746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6a79-f0ea-4dbd-af97-e53707fae6fd admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.840021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6a79-f0ea-4dbd-af97-e53707fae6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.840291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a6a79-f0ea-4dbd-af97-e53707fae6fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.840921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a6a79-f0ea-4dbd-af97-e53707fae6fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.842544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a6a79-f0ea-4dbd-af97-e53707fae6fd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.853472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1027/2043] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:34:30.857501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:30.867563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8475d38-17e5-469a-af7a-5e2fbaa70c94 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:30.873042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:30.874106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.874380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.874649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] PATH_INFO: `/v3/projects/1628052c47a64f03bdf9238a5a4d08e7/users/100619806db54a1887d9eec477de5ce2/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.890726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1628052c47a64f03bdf9238a5a4d08e7, user_id=100619806db54a1887d9eec477de5ce2, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:30.893970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:30.902520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:30.907578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1017/2044] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:30 2022] PUT /identity/v3/projects/1628052c47a64f03bdf9238a5a4d08e7/users/100619806db54a1887d9eec477de5ce2/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:34:30.918791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:30.919078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:30.919349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:30.946412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] MFA Rules not processed for user `100619806db54a1887d9eec477de5ce2`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:30.947613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.011024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1028/2045] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:31.015558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.076055 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bf30c-a879-42c9-b5a2-d254a4e8b5f5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.081175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.081817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.082091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.082362 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.082987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.084878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.110194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1018/2046] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:31.114259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.124947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dbdb02-23ef-4c37-8020-4dc7681eb883 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.159730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.184396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.184998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.185275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.185577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.186206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.188116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.209590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1029/2047] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:31.213540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.223770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d7bf6f-b8f1-4b13-9b19-e29bd0f44dc7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.228332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.228914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.229189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.229498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.230129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.231713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.242023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1019/2048] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:31.246026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.256432 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e567964-01d8-4c18-b26b-58fcd0f0d165 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.261032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ac6e8-651f-49ff-b324-09c9909224db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.261658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ac6e8-651f-49ff-b324-09c9909224db admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.261937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ac6e8-651f-49ff-b324-09c9909224db admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.262205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ac6e8-651f-49ff-b324-09c9909224db admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.262844 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ac6e8-651f-49ff-b324-09c9909224db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.265157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ac6e8-651f-49ff-b324-09c9909224db admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.278961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1030/2049] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:31.281273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.291621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e21e95-d36f-46d8-94d2-cdd0fff1bcc3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.297099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.297770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.298046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.298336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] PATH_INFO: `/v3/projects/6c884fffbb5d4761884835efb028fc99/users/8a4b14382a274d0da7816ba487fc0f93/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.312953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c884fffbb5d4761884835efb028fc99, user_id=8a4b14382a274d0da7816ba487fc0f93, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.316201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.330297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1020/2050] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:31 2022] PUT /identity/v3/projects/6c884fffbb5d4761884835efb028fc99/users/8a4b14382a274d0da7816ba487fc0f93/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:31.340914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.341369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.341644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.374922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] MFA Rules not processed for user `8a4b14382a274d0da7816ba487fc0f93`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:31.376139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.439445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1031/2051] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:31.444686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.510933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe211-1013-4958-b61c-ddad7335b44f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.515759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.516380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.516657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.516930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.517610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.519877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.552619 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1021/2052] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:31.553868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.565923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2a34b-db08-4d1b-b01d-6589c37ac871 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.628487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.629173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.629415 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.629687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.630313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.632244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.653054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1032/2053] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:34:31.657154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.667603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f67fb7-12a7-4b8c-82f4-16b9d7c46b63 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.672164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.672754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.673032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.673319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.674492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.676085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.686381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1022/2054] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:31.691043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.700848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad9897-6a10-452e-9173-d1b0653f61d4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.705581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.706297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.706574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.706881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] PATH_INFO: `/v3/projects/4d5ecda882e546e4acbbafd2d9bedfbd/users/8c0f4e23200341e4bd488035a2ff0dcf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.721982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5ecda882e546e4acbbafd2d9bedfbd, user_id=8c0f4e23200341e4bd488035a2ff0dcf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.724679 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.738525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1033/2055] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:31 2022] PUT /identity/v3/projects/4d5ecda882e546e4acbbafd2d9bedfbd/users/8c0f4e23200341e4bd488035a2ff0dcf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:31.742302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.752336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb6c69-7d61-4b8f-8d6a-97340bd736f0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.771763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.812001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d9967-d075-4d11-b088-889be63f2e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.812719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9967-d075-4d11-b088-889be63f2e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.812880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9967-d075-4d11-b088-889be63f2e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.813154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d9967-d075-4d11-b088-889be63f2e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.813842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9967-d075-4d11-b088-889be63f2e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.815497 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d9967-d075-4d11-b088-889be63f2e3e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.825870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1023/2056] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:34:31 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:34:31.830090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:31.840829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9752477e-2c22-42e0-8d48-47fdcc70413f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:31.845404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:31.846596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.846872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.847170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] PATH_INFO: `/v3/projects/4d5ecda882e546e4acbbafd2d9bedfbd/users/8c0f4e23200341e4bd488035a2ff0dcf/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.849057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5ecda882e546e4acbbafd2d9bedfbd, user_id=8c0f4e23200341e4bd488035a2ff0dcf, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:31.851732 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:31.854889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:31.864568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1034/2057] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:34:31 2022] PUT /identity/v3/projects/4d5ecda882e546e4acbbafd2d9bedfbd/users/8c0f4e23200341e4bd488035a2ff0dcf/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:31.876188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:31.876487 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:31.876769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:31.911817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] MFA Rules not processed for user `8c0f4e23200341e4bd488035a2ff0dcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:31.913061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:31.988115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1024/2058] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:31.999749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:32.001516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:32.063183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1efc-57dd-4463-b0d0-951de4c6775c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:32.067069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3770e-8d83-4101-b2dd-d2454fefe46e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:32.067664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3770e-8d83-4101-b2dd-d2454fefe46e service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:32.067939 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3770e-8d83-4101-b2dd-d2454fefe46e service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:32.068210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3770e-8d83-4101-b2dd-d2454fefe46e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:32.126353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3770e-8d83-4101-b2dd-d2454fefe46e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:32.129278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3770e-8d83-4101-b2dd-d2454fefe46e service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:32.138828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1035/2059] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:37.931521 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:37.933594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:37.950210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42a3c6-deb0-4b67-932e-49f2610c7c92 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:37.954836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:37.955792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:37.956423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:37.957032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:38.068114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:38.068743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:38.068743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:38.082655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:38.084100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:38.105709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1025/2060] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:34:38.111128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:38.113347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:38.218089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1036/2061] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:34:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:34:38.233221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:38.237703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:38.347297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:38.380314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95307e56-c151-4b57-aee2-6ae2da8729fc service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:38.388052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:38.388670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:38.388947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:38.389217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:38.398589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:38.400285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:38.413276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1026/2062] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:34:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:34:39.190865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:39.209371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee65e6-9f5a-46f0-84d9-4daedab2ff28 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:39.213593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf4d98-48ea-4ef2-9442-ed5160cb43c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:39.215344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf4d98-48ea-4ef2-9442-ed5160cb43c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:39.215632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf4d98-48ea-4ef2-9442-ed5160cb43c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:39.215906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf4d98-48ea-4ef2-9442-ed5160cb43c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:39.227438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf4d98-48ea-4ef2-9442-ed5160cb43c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:39.229447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf4d98-48ea-4ef2-9442-ed5160cb43c8 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:39.242114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1037/2063] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:44.558898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:44.665013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb287b6f-e500-4749-b83f-3be32464ded7 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:44.673662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:44.674399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:44.674688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:44.674970 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] PATH_INFO: `/v3/users/100619806db54a1887d9eec477de5ce2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:44.676306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=100619806db54a1887d9eec477de5ce2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:44.678307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:44.749353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100619806db54a1887d9eec477de5ce2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:44.749353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100619806db54a1887d9eec477de5ce2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:44.749353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100619806db54a1887d9eec477de5ce2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:44.767608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100619806db54a1887d9eec477de5ce2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:44.777416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100619806db54a1887d9eec477de5ce2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:44.802401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1027/2064] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:34:44 2022] DELETE /identity/v3/users/100619806db54a1887d9eec477de5ce2 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:44.806866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:44.807383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:44.807486 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:44.828588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:34:44.829961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:44.876792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:44.900147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1038/2065] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:34:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:34:44.910131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:44.911612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:44.979192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f9d07d-2805-4425-a41b-30d84eeec296 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:44.982886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:44.983511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:44.983920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:44.984225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:44.992961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:44.996993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:44.998086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:45.010977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1028/2066] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:34:45.577130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:45.578807 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:45.608978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdabdf-8afc-4f97-b544-48b00b390d53 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:45.621296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:45.622060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:45.622454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:45.622767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] PATH_INFO: `/v3/projects/1628052c47a64f03bdf9238a5a4d08e7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:45.624156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1628052c47a64f03bdf9238a5a4d08e7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:45.626346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:45.679087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1628052c47a64f03bdf9238a5a4d08e7', 'request_id': 'req-6256448c-e913-4a60-9bb7-8e13656347b3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.685978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1628052c47a64f03bdf9238a5a4d08e7', 'request_id': 'req-6256448c-e913-4a60-9bb7-8e13656347b3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.695020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1628052c47a64f03bdf9238a5a4d08e7', 'request_id': 'req-6256448c-e913-4a60-9bb7-8e13656347b3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.706943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] The token cache is being invalidate because project 1628052c47a64f03bdf9238a5a4d08e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:34:45.707900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.708560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.710574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1039/2067] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:34:45 2022] DELETE /identity/v3/projects/1628052c47a64f03bdf9238a5a4d08e7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:45.714731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:45.725615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cfb20-6da8-4503-8a1a-51a6be67b63e service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:45.783036 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:45.783802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:45.784079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:45.784348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] PATH_INFO: `/v3/users/8a4b14382a274d0da7816ba487fc0f93` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:45.785947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a4b14382a274d0da7816ba487fc0f93)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:45.788107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:45.870090 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a4b14382a274d0da7816ba487fc0f93'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.871410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a4b14382a274d0da7816ba487fc0f93'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.872049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a4b14382a274d0da7816ba487fc0f93'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.882233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a4b14382a274d0da7816ba487fc0f93'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.914417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a4b14382a274d0da7816ba487fc0f93'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:45.948389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1029/2068] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:34:45 2022] DELETE /identity/v3/users/8a4b14382a274d0da7816ba487fc0f93 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:46.289739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:46.291402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:46.307153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6256448c-e913-4a60-9bb7-8e13656347b3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:46.381890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:46.382515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:46.382789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:46.383056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] PATH_INFO: `/v3/projects/6c884fffbb5d4761884835efb028fc99` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:46.384150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c884fffbb5d4761884835efb028fc99)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:46.386495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:46.455378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c884fffbb5d4761884835efb028fc99', 'request_id': 'req-dfe09f95-b5e2-40be-86d5-4e08d145156c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.462328 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c884fffbb5d4761884835efb028fc99', 'request_id': 'req-dfe09f95-b5e2-40be-86d5-4e08d145156c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.470335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c884fffbb5d4761884835efb028fc99', 'request_id': 'req-dfe09f95-b5e2-40be-86d5-4e08d145156c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.475698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:46.482984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] The token cache is being invalidate because project 6c884fffbb5d4761884835efb028fc99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:34:46.483361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.484107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.486027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1040/2069] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:34:46 2022] DELETE /identity/v3/projects/6c884fffbb5d4761884835efb028fc99 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:46.489978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:46.491538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:46.507294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bb7c4b-d3aa-476d-b8e9-7b221ab78f00 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:46.574766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:46.575517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:46.575913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:46.576287 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] PATH_INFO: `/v3/users/8c0f4e23200341e4bd488035a2ff0dcf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:46.577576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0f4e23200341e4bd488035a2ff0dcf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:46.579714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:46.598865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:46.652822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0f4e23200341e4bd488035a2ff0dcf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.653638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0f4e23200341e4bd488035a2ff0dcf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.654192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0f4e23200341e4bd488035a2ff0dcf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.661770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0f4e23200341e4bd488035a2ff0dcf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.675941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1ed04bbd-b385-4bd5-81a9-a65b025cb096 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0f4e23200341e4bd488035a2ff0dcf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:46.694552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1030/2070] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:34:46 2022] DELETE /identity/v3/users/8c0f4e23200341e4bd488035a2ff0dcf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:47.052126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:34:47.053709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:34:47.067406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe09f95-b5e2-40be-86d5-4e08d145156c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:34:47.163326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:34:47.164740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:34:47.165442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:34:47.165954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] PATH_INFO: `/v3/projects/4d5ecda882e546e4acbbafd2d9bedfbd` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:34:47.167763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5ecda882e546e4acbbafd2d9bedfbd)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:34:47.170421 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:34:47.296232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5ecda882e546e4acbbafd2d9bedfbd', 'request_id': 'req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:47.318484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5ecda882e546e4acbbafd2d9bedfbd', 'request_id': 'req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:47.328820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5ecda882e546e4acbbafd2d9bedfbd', 'request_id': 'req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:47.341735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] The token cache is being invalidate because project 4d5ecda882e546e4acbbafd2d9bedfbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:34:47.342054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:47.342736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ab49c31-3afa-4775-b6ca-60ac28fdcd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:34:47.344455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1041/2071] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:34:47 2022] DELETE /identity/v3/projects/4d5ecda882e546e4acbbafd2d9bedfbd => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:34:56.690305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=9,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:34:57.338289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:35:00.420554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:00.421024 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:00.421163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:00.440153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] MFA Rules not processed for user `4b702145173b471e880425c0703b4d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:00.441409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:00.494549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1031/2072] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:35:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:10.487468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=25 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:35:29.015325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96108af3-02e8-47f6-9f05-c1a9da93c505 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:29.015889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96108af3-02e8-47f6-9f05-c1a9da93c505 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:29.015986 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96108af3-02e8-47f6-9f05-c1a9da93c505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:29.035355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-96108af3-02e8-47f6-9f05-c1a9da93c505 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:29.036564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-96108af3-02e8-47f6-9f05-c1a9da93c505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:29.094251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1042/2073] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:35:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:29.119311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:29.121580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:29.199511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d209b18-2008-4bbc-8db9-55564dcaa52c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:29.218816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:29.219722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:29.220001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:29.220275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:29.229884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:29.231477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:29.239744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1032/2074] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:35:29.765997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe8eaa-20e3-4b2d-9684-8c6df282651a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:29.766285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe8eaa-20e3-4b2d-9684-8c6df282651a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:29.766559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe8eaa-20e3-4b2d-9684-8c6df282651a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:29.786513 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-95fe8eaa-20e3-4b2d-9684-8c6df282651a None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:29.787717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-95fe8eaa-20e3-4b2d-9684-8c6df282651a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:29.795779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1043/2075] 10.222.0.17 () {58 vars in 967 bytes} [Wed Oct 12 12:35:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:29.809231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:29.819375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10bf801-3ec1-4278-9827-a35599368231 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:29.824988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:29.825600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:29.825877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:29.826144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:29.834172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:29.835737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:29.844698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1033/2076] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:35:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 12 12:35:30.082994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25d508-715b-49bb-905b-091dd4bca69f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:30.083435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25d508-715b-49bb-905b-091dd4bca69f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:30.083526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae25d508-715b-49bb-905b-091dd4bca69f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:30.103557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ae25d508-715b-49bb-905b-091dd4bca69f None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:30.104805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae25d508-715b-49bb-905b-091dd4bca69f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:30.111880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1044/2077] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:35:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:30.124788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:30.136167 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c999e807-8370-47de-8c89-7e08e281e7db service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:30.141134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:30.141752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:30.142029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:30.142297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:30.151140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:30.152691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:30.160434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1034/2078] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:35:30.515704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed9d6f-2192-4d2e-b5df-c5208744537b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:30.516007 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed9d6f-2192-4d2e-b5df-c5208744537b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:30.516441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed9d6f-2192-4d2e-b5df-c5208744537b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:30.538656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-66ed9d6f-2192-4d2e-b5df-c5208744537b None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:30.540770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-66ed9d6f-2192-4d2e-b5df-c5208744537b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:30.549035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1045/2079] 10.222.0.17 () {58 vars in 967 bytes} [Wed Oct 12 12:35:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:30.563932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:30.574845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df14d8f2-d004-40e7-9f20-ba58a17b0e3e service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:30.579760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:30.580339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:30.580615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:30.580883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:30.589482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:30.591067 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:30.598826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1035/2080] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:35:33.055615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811681-5816-4f7a-aa64-1f0837046df7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:33.056117 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811681-5816-4f7a-aa64-1f0837046df7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:33.056227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c811681-5816-4f7a-aa64-1f0837046df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:33.075220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4c811681-5816-4f7a-aa64-1f0837046df7 None None] MFA Rules not processed for user `fd90f4df65dd43e4814a60c1955114fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:33.076403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4c811681-5816-4f7a-aa64-1f0837046df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:33.130736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1046/2081] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:33.781335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:33.782842 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:33.850509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784952f-11e0-4000-b389-670e325a64d8 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:33.855662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:33.856289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:33.856595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:33.856864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] PATH_INFO: `/v3/users/298ebaa6478544dba2df00dabb44ad46` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:33.858303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=298ebaa6478544dba2df00dabb44ad46)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:33.859912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:33.910794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '298ebaa6478544dba2df00dabb44ad46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:33.913734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '298ebaa6478544dba2df00dabb44ad46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:33.914410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '298ebaa6478544dba2df00dabb44ad46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:33.924140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:35:33.928411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '298ebaa6478544dba2df00dabb44ad46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:33.938912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '298ebaa6478544dba2df00dabb44ad46'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:33.958821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1036/2082] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:35:33 2022] DELETE /identity/v3/users/298ebaa6478544dba2df00dabb44ad46 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:35:33.964201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:33.964565 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:33.964936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:33.986237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:35:33.987559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:34.051579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1047/2083] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:35:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:35:34.061561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:34.062985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:34.123133 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4860e6-4fe2-496d-be63-142db7202bc5 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:34.126969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:34.127576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:34.127846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:34.128111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:34.139576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:34.141278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:34.151967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1037/2084] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:35:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:35:34.352525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:34.356361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:34.388875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9830154-b6a4-4e84-b26b-7d7ba0196027 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:34.396509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:34.397471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:34.397793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:34.398152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] PATH_INFO: `/v3/projects/af1b620b3ebe475b8a6d82e4cb5d5097` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:34.401523 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af1b620b3ebe475b8a6d82e4cb5d5097)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:34.404325 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:34.443723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:35:34.467184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af1b620b3ebe475b8a6d82e4cb5d5097', 'request_id': 'req-f962789e-f3a7-410b-b977-fb11319187f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:34.474884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af1b620b3ebe475b8a6d82e4cb5d5097', 'request_id': 'req-f962789e-f3a7-410b-b977-fb11319187f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:34.483741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af1b620b3ebe475b8a6d82e4cb5d5097', 'request_id': 'req-f962789e-f3a7-410b-b977-fb11319187f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:34.501591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] The token cache is being invalidate because project af1b620b3ebe475b8a6d82e4cb5d5097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:35:34.502023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:34.502916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:35:34.517404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1048/2085] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:35:34 2022] DELETE /identity/v3/projects/af1b620b3ebe475b8a6d82e4cb5d5097 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:35:43.602765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:43.604349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:43.615656 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7a798-b76c-4953-88ff-8abd38475bc6 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:43.670903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:43.671510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:43.671785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:43.672053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:43.674046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:43.748113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:43.758168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:43.769917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1038/2086] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:35:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:35:44.497665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:35:46.926947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:46.929687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:46.948354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f962789e-f3a7-410b-b977-fb11319187f7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:46.965347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:46.966238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:46.966590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:46.966936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:46.969729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:47.046362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:47.047879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:47.063735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1049/2087] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:35:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:35:53.765023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:35:55.366103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:55.367572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:55.378534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1e545-cde9-4e78-8fe8-a80cc694b7ed service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:55.382125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:55.382704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:55.382980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:55.383275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:55.385081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:55.463525 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:55.464961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248b7c1b-d037-47c4-b7b7-bc2b7ddb8262 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:55.475969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1039/2088] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:35:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:35:56.657740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:35:56.666601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5121999-a6f3-4d10-a2b8-f46ab43831e7 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:35:56.670229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:35:56.670852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:35:56.671130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:35:56.671399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:35:56.673123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:35:56.686102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:35:56.687508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:35:56.698587 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1050/2089] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:35:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:01.020136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:01.020618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:01.020737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:01.039840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] MFA Rules not processed for user `383004527ce84e4f9fc40760a040936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:36:01.041565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:01.094408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1040/2090] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:36:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:36:06.693335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=36 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:11.087552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=59 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:13.677178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:13.678649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:13.692678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887407ec-f8fd-4ff0-98f0-6b8439064b7d service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:13.698772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:13.699361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:13.699636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:13.699904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:13.701598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:13.759499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:13.761112 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:13.770139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1051/2091] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:14.448899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:14.450314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:14.466814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4eb0b9-ff95-4b3f-99a7-54f23cc9a4f1 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:14.472144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:14.472731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:14.473011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:14.473283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:14.474957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:14.485244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:14.486834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:14.496178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1041/2092] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:21.381772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:21.391676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251bf221-9efc-4998-82ea-5e9acf49549b service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:21.395207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:21.395822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:21.396095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:21.396361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:21.398111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:21.407931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:21.409427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:21.417680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1052/2093] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:24.491468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:29.764777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:29.777010 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f3b8d-cff2-49e3-82ef-746f7eca8150 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:29.783195 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:29.783795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:29.784203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:29.784478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:29.786441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:29.800982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:29.802627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:29.814876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1042/2094] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:30.018440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:30.035909 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45e98a-d46c-4380-a436-104466c5ae99 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:30.043999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:30.045396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:30.045869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:30.046312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:30.049902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:30.066147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:30.067765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:30.080696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1053/2095] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:39.810802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:40.076139 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=40 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:36:58.191725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:58.193331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:58.256094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:58.257522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:58.277668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293fae57-140c-4222-a7a7-76ddb5298f1a service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:58.282981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:58.283632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:58.284131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:58.284263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:58.285947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:36:58.299803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:58.301421 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:58.312082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1043/2096] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:36:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:36:58.331003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69407a32-72bd-421d-9997-fcfd3f743b89 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:58.335516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:58.336137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:58.336414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:58.336736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] PATH_INFO: `/v3/users/fd90f4df65dd43e4814a60c1955114fa` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:58.338945 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd90f4df65dd43e4814a60c1955114fa)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:58.340601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:58.386496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd90f4df65dd43e4814a60c1955114fa'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.393740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd90f4df65dd43e4814a60c1955114fa'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.394349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd90f4df65dd43e4814a60c1955114fa'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.401770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd90f4df65dd43e4814a60c1955114fa'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.402399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd90f4df65dd43e4814a60c1955114fa'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.420168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1054/2097] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:36:58 2022] DELETE /identity/v3/users/fd90f4df65dd43e4814a60c1955114fa => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:36:58.587081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:36:58.598640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5f1a40-00ae-4b00-886a-8b1b6052b483 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:36:58.656066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:36:58.656682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:36:58.656957 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:36:58.657224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] PATH_INFO: `/v3/projects/3f584855911f47de84309fc26b2e25ee` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:36:58.658343 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f584855911f47de84309fc26b2e25ee)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:36:58.660238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:36:58.705690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f584855911f47de84309fc26b2e25ee', 'request_id': 'req-3d1e4581-3df7-48e6-b803-22fa9e2d528d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.712569 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f584855911f47de84309fc26b2e25ee', 'request_id': 'req-3d1e4581-3df7-48e6-b803-22fa9e2d528d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.719534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f584855911f47de84309fc26b2e25ee', 'request_id': 'req-3d1e4581-3df7-48e6-b803-22fa9e2d528d'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.729893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] The token cache is being invalidate because project 3f584855911f47de84309fc26b2e25ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:36:58.730181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.730812 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-3d1e4581-3df7-48e6-b803-22fa9e2d528d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:36:58.732286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1044/2098] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:36:58 2022] DELETE /identity/v3/projects/3f584855911f47de84309fc26b2e25ee => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:37:08.415992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:08.726709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:32.298529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:37:32.299958 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:32.371806 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456660b-31fa-4b90-a680-4fb65958fdea admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:37:32.432783 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:37:32.433697 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:32.434206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:32.434701 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:32.436517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:32.492218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:37:32.493820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8fd4eb-9c6d-4e04-839c-caad9216d1cc service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:37:32.502922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1055/2099] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:37:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:37:42.498757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:37:56.278776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:37:56.279313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:37:56.279313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:37:56.299178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] MFA Rules not processed for user `76e3059b01884215becf97694c3b7027`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:37:56.300371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:37:56.307142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1045/2100] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:37:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:06.293700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:25.830872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:25.831914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:25.831914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:25.851731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] MFA Rules not processed for user `383004527ce84e4f9fc40760a040936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:25.852937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:25.907744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1056/2101] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:38:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:26.217625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:26.219070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:26.275088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b075c-a817-498f-8960-a4e3534f9451 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:26.278612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:26.279208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.279482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.279749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.281457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:26.290874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:26.292281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:26.300292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1046/2102] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:26.618928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:26.620424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:26.692717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c21119-2dba-4153-afa9-d435301e2608 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:26.697696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:26.698299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:26.698577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:26.698846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:26.700554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:26.711813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:26.713409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f6d39-c30c-46ed-a48a-ee9f8ee905ed service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:26.723256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1057/2103] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:35.662894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:35.664473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:35.682340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8205a-8ab7-426c-9707-ed7da6b471ba service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:35.687766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:35.688355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:35.688631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:35.688901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:35.690697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:35.703059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:35.704630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:35.716033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1047/2104] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:36.718960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=57 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:45.711665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=32 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:47.440314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.440791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.440896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.459770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] MFA Rules not processed for user `4b702145173b471e880425c0703b4d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:38:47.460972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.519119 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1058/2105] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:38:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:38:47.836822 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:38:47.838295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.849959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6e323-bb86-47a4-a009-4d75e3c601ce service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:38:47.853443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:38:47.854026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:38:47.854306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:38:47.854578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:38:47.856218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:38:47.866217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:38:47.867687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:38:47.875867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1048/2106] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:38:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:38:57.508720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=25 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:38:57.872232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:08.885804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:08.887357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:08.979409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d5d29c-4909-42bc-9bde-aa02c86758a4 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:08.985532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:08.986155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:08.986431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:08.986725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:08.988402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:09.046582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:09.048182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc5a11-42a8-4b01-8670-4e62fb936218 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:09.056917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1059/2107] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:19.052811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=62 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:21.084815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:21.184826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00cbef9-1e9c-45b2-bd17-388fec123195 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:21.193845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:21.194707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:21.195804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:21.196314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:21.198606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:21.246914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:21.249098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:21.265803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1049/2108] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:22.821017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:22.821555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:22.821922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:22.865762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:22.867567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:22.934540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1060/2109] 10.222.0.17 () {58 vars in 967 bytes} [Wed Oct 12 12:39:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:22.989081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:22.990692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:23.074911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d14ec-4627-4aa3-9f4c-a4e2bb9e4ff3 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:23.082282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4244086-07e2-4474-b899-f99e139f4779 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:23.083122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244086-07e2-4474-b899-f99e139f4779 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.083502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244086-07e2-4474-b899-f99e139f4779 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.083984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4244086-07e2-4474-b899-f99e139f4779 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:23.097494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4244086-07e2-4474-b899-f99e139f4779 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:23.099116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4244086-07e2-4474-b899-f99e139f4779 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:23.116211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1050/2110] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:39:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:39:23.562013 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:23.583194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4106e16-795f-4c0c-807e-66a0027b974f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:23.588211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:23.589671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:23.590134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:23.590402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:23.592168 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:23.708745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:23.710543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:23.741523 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1061/2111] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:26.043162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:26.043649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:26.043744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:26.086551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] MFA Rules not processed for user `abbc8a34ac8f44328596102c35e184dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:26.087814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:26.186286 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1051/2112] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:39:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:31.926790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:31.928238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:31.942365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881fa4a-f5e8-400b-bf94-9e6426e8ce22 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:31.947448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:31.948043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:31.948434 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:31.948712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:31.950430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:31.962795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:31.964376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:31.974563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1062/2113] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:32.013538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.015129 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.029785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6313590-5d1c-41d4-bef4-79369adbd91b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.038161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.039144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.039403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.039742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] PATH_INFO: `/v3/users/d1fbfe1a7b4347a6a28241a8e8d815e0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.053696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1fbfe1a7b4347a6a28241a8e8d815e0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.053696 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.089321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:32.096626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1fbfe1a7b4347a6a28241a8e8d815e0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.097295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1fbfe1a7b4347a6a28241a8e8d815e0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.097988 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1fbfe1a7b4347a6a28241a8e8d815e0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.105277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1fbfe1a7b4347a6a28241a8e8d815e0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.115881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1fbfe1a7b4347a6a28241a8e8d815e0'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.137087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1052/2114] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:32 2022] DELETE /identity/v3/users/d1fbfe1a7b4347a6a28241a8e8d815e0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:32.318691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.330331 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2c787-7f83-4e70-94b6-60adf0186a45 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.370186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:32.389264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.389915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.390188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.390454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] PATH_INFO: `/v3/projects/ded843c641e84227bfadd4d9d590317c` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.391543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ded843c641e84227bfadd4d9d590317c)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.393476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.439790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ded843c641e84227bfadd4d9d590317c', 'request_id': 'req-d4486fb8-0296-49b2-8cb1-ba9711322de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.446871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ded843c641e84227bfadd4d9d590317c', 'request_id': 'req-d4486fb8-0296-49b2-8cb1-ba9711322de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.453967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ded843c641e84227bfadd4d9d590317c', 'request_id': 'req-d4486fb8-0296-49b2-8cb1-ba9711322de9'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.463925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] The token cache is being invalidate because project ded843c641e84227bfadd4d9d590317c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:32.464207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.464819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.466296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1063/2115] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:32 2022] DELETE /identity/v3/projects/ded843c641e84227bfadd4d9d590317c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:32.470303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.471722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.484834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abf725-f38e-4de2-bf73-dd15d49582fe admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.540704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.541342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.541645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.541898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] PATH_INFO: `/v3/users/0ff542c9edb647019d9b38eee76e68bf` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.543012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff542c9edb647019d9b38eee76e68bf)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.544926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.590294 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff542c9edb647019d9b38eee76e68bf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.590862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff542c9edb647019d9b38eee76e68bf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.591471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff542c9edb647019d9b38eee76e68bf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.598273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff542c9edb647019d9b38eee76e68bf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.605620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff542c9edb647019d9b38eee76e68bf'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.623749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1053/2116] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:32 2022] DELETE /identity/v3/users/0ff542c9edb647019d9b38eee76e68bf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:32.794940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:32.796378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:32.809733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4486fb8-0296-49b2-8cb1-ba9711322de9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:32.873704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:32.874255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:32.874791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:32.875064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] PATH_INFO: `/v3/projects/14383ebc601b4a16aab303b4c24925f5` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:32.876264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14383ebc601b4a16aab303b4c24925f5)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:32.878176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:32.925698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14383ebc601b4a16aab303b4c24925f5', 'request_id': 'req-614b2826-0d09-45cf-a686-a0d6f339dc96'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.932646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14383ebc601b4a16aab303b4c24925f5', 'request_id': 'req-614b2826-0d09-45cf-a686-a0d6f339dc96'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.940273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14383ebc601b4a16aab303b4c24925f5', 'request_id': 'req-614b2826-0d09-45cf-a686-a0d6f339dc96'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.948562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:32.952753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] The token cache is being invalidate because project 14383ebc601b4a16aab303b4c24925f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:32.953483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.954370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:32.956202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1064/2117] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:32 2022] DELETE /identity/v3/projects/14383ebc601b4a16aab303b4c24925f5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:35.156020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:35.157478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:35.197248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:35.227572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb13caa-76fe-45f9-a88b-91d1c862f486 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:35.275387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:35.276023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:35.276302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:35.276600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:35.278377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:35.341384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:35.342819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:35.353383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1054/2118] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:39.069911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.071498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.085184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b2826-0d09-45cf-a686-a0d6f339dc96 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.089919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.090538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.090812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.091078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] PATH_INFO: `/v3/users/76e3059b01884215becf97694c3b7027` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.092177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e3059b01884215becf97694c3b7027)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.094937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.143542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e3059b01884215becf97694c3b7027'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.150419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e3059b01884215becf97694c3b7027'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.151206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e3059b01884215becf97694c3b7027'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.158304 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e3059b01884215becf97694c3b7027'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.158927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-57607342-7f9f-4bb6-9621-af6238fbff2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e3059b01884215becf97694c3b7027'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.176161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1065/2119] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:39:39 2022] DELETE /identity/v3/users/76e3059b01884215becf97694c3b7027 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:39.353545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.366824 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2275d-a6ec-4476-8085-1cfecdd54df2 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.423428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.423974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.424251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.424520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] PATH_INFO: `/v3/projects/63cab3964ec14361b50e591258284ab0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.425655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63cab3964ec14361b50e591258284ab0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.427545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.462210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:39.474753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63cab3964ec14361b50e591258284ab0', 'request_id': 'req-b10a3bb2-1af5-4ab8-b603-99419bdee439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.482475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63cab3964ec14361b50e591258284ab0', 'request_id': 'req-b10a3bb2-1af5-4ab8-b603-99419bdee439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.488810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63cab3964ec14361b50e591258284ab0', 'request_id': 'req-b10a3bb2-1af5-4ab8-b603-99419bdee439'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.499345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] The token cache is being invalidate because project 63cab3964ec14361b50e591258284ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:39:39.499885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.500503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b10a3bb2-1af5-4ab8-b603-99419bdee439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:39:39.502125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1055/2120] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:39:39 2022] DELETE /identity/v3/projects/63cab3964ec14361b50e591258284ab0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:39:39.507175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.507462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.507811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.565130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:39.566378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.625764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1066/2121] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:39.633876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.634161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.634436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.654776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:39.655991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:39.663945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1056/2122] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:39.668364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.678113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421674a9-7a19-47f1-bdbd-4ad564deb283 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.682638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.683254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.683530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.683801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.684439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:39:39.684762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.686385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.698223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1067/2123] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:39:39.702344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.712693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85473947-4444-4e96-bdb4-73a3759bf65f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.717386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.717981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.718322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.718504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.719235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.720997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.748723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1057/2124] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:39.753244 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.763602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce9e2b2-428e-44ba-ba0e-ef8ee037aa5b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.791010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:39.824010 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.824623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.824908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.825188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.825858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.827792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.852897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1068/2125] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:39:39.857025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.868332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c2c0c-bb8f-4002-987b-cfa55d79267b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.873126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.873789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.874070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.874345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.875257 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.876887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.888143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1058/2126] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:39.891934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.902351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129601d-2529-458a-ba07-3e7bad3ef915 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.907153 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a46b6e-1ead-479b-b64d-cb3573fddfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.907752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46b6e-1ead-479b-b64d-cb3573fddfce admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.908031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46b6e-1ead-479b-b64d-cb3573fddfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.908336 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a46b6e-1ead-479b-b64d-cb3573fddfce admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.909009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46b6e-1ead-479b-b64d-cb3573fddfce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.910646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a46b6e-1ead-479b-b64d-cb3573fddfce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.921472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1069/2127] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:39:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:39:39.925692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:39.936413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bd035-17b4-44bf-82ef-1ea3307bc675 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:39.941150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:39.941837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.942115 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.942387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] PATH_INFO: `/v3/projects/e57c943e5713455781e1799934cef6bc/users/d0010af061ae4490a6855639a6825cac/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:39.958047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e57c943e5713455781e1799934cef6bc, user_id=d0010af061ae4490a6855639a6825cac, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:39.961332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:39.974565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1059/2128] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:39:39 2022] PUT /identity/v3/projects/e57c943e5713455781e1799934cef6bc/users/d0010af061ae4490a6855639a6825cac/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:39:39.980472 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8925-df72-4992-9597-6b57166a2731 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:39.980753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8925-df72-4992-9597-6b57166a2731 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:39.981024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a8925-df72-4992-9597-6b57166a2731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.000709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-289a8925-df72-4992-9597-6b57166a2731 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:40.001947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-289a8925-df72-4992-9597-6b57166a2731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:40.062868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1070/2129] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:39:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:40.072999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:40.074779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:40.132178 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381d4fd-9b43-478e-be40-d389f4721d3f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:40.135850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:40.136452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:40.136730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:40.136996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:40.146143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:40.147576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:40.157882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1060/2130] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:44.509355 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db3c08-9b8e-4979-891d-3a9416fd73fb None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:44.509859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db3c08-9b8e-4979-891d-3a9416fd73fb None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:44.509859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db3c08-9b8e-4979-891d-3a9416fd73fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:44.543896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-47db3c08-9b8e-4979-891d-3a9416fd73fb None None] MFA Rules not processed for user `d0010af061ae4490a6855639a6825cac`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:44.545240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-47db3c08-9b8e-4979-891d-3a9416fd73fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:44.615758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1071/2131] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:39:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:44.631224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:39:44.633375 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:44.695291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:44.714174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8eefc2-faf8-4f0d-b35f-ba23cf931b46 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:39:44.719853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c33f9-8f4b-405e-a94f-ba9e77334c2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:39:44.720464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c33f9-8f4b-405e-a94f-ba9e77334c2b service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:44.720742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c33f9-8f4b-405e-a94f-ba9e77334c2b service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:44.721016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c33f9-8f4b-405e-a94f-ba9e77334c2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:44.730163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c33f9-8f4b-405e-a94f-ba9e77334c2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:39:44.731750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34c33f9-8f4b-405e-a94f-ba9e77334c2b service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:39:44.741858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1061/2132] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:39:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:39:46.013735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:39:46.014199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:39:46.014316 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:39:46.033608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] MFA Rules not processed for user `d0010af061ae4490a6855639a6825cac`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:39:46.034931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:39:46.042521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1072/2133] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:39:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:39:54.737599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:39:56.028944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:40:15.442951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:40:15.443431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:40:15.443626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:40:15.463220 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] MFA Rules not processed for user `383004527ce84e4f9fc40760a040936e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:40:15.464558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:40:15.517772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1062/2134] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:40:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:40:25.511130 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=25 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:09.401757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:09.403155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:09.480428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7674b3-63e4-4515-94d7-b58d1c56bfc2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:09.484003 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:09.484602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:09.484950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:09.485228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:09.487351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:09.548243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:09.549640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:09.559342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1073/2135] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:09.725340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:09.726189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:09.740199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7b89d-a431-468e-8366-d4e4151b25f0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:09.744670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:09.745323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:09.751028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:09.751028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:09.751028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:09.762753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:09.764200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800e48a7-73fe-4676-93be-43aa84d8560d service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:09.775749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1063/2136] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:14.251915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:14.260722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa26fbc-aec3-4a20-88af-d02731459a05 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:14.265453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:14.266103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:14.266369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:14.266633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:14.268369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:14.348847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:14.350333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:14.368647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1074/2137] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:19.770161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:22.684741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:22.685404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:22.685512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:22.704547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] MFA Rules not processed for user `4b702145173b471e880425c0703b4d7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:22.705888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:22.712641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1064/2138] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:22.749276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:22.754640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:22.758236 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3d204-0067-4d61-a966-dea6b347e130 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:22.762182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:22.763483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:22.763761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:22.764033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:22.765835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:22.779204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:22.780654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:22.791910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1075/2139] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:25.786504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:25.788081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:25.801854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0027d9-4f98-4c06-bf55-8241b9286829 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:25.805735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:25.806635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:25.807122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:25.807581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:25.811015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:25.822164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:25.823713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:25.835538 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1065/2140] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:32.786625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:35.829880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=9 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:41.043941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:41.057561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034c9ab6-c7a7-40cb-b67f-ac292eaca023 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:41.064925 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:41.065595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:41.067280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:41.067561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:41.069973 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:41.086393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:41.088334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:41.112771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1076/2141] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:47.133018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:47.142373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd82655a-6a1c-48ff-ac2e-8984b79f3d25 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:47.146118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:47.146778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:47.147058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:47.147326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:47.149019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:47.161997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:47.163402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:47.173566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1066/2142] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:41:48.978323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:48.991576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb73ab-1bdb-49c2-b03b-e4aa399b5c97 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:48.996572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:48.997426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:48.998045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:48.998499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:49.000831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:49.094653 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:49.096311 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453412f3-e1d4-45df-b3b7-bcea4930bf6d service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:49.107229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1077/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:41:50.770462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:50.772040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:50.785928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f9c8-e2f6-4b0a-97a3-88628de32bd5 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:50.803611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:50.805233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:50.805766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:50.806255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] PATH_INFO: `/v3/users/abbc8a34ac8f44328596102c35e184dc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:50.807835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abbc8a34ac8f44328596102c35e184dc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:50.811081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:50.867353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df777-0066-4b1c-883e-9c6762981560 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:50.868350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df777-0066-4b1c-883e-9c6762981560 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:50.868626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173df777-0066-4b1c-883e-9c6762981560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:50.874023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abbc8a34ac8f44328596102c35e184dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:50.874742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abbc8a34ac8f44328596102c35e184dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:50.875326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abbc8a34ac8f44328596102c35e184dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:50.883392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abbc8a34ac8f44328596102c35e184dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:50.892779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abbc8a34ac8f44328596102c35e184dc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:50.913422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1067/2144] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:41:50 2022] DELETE /identity/v3/users/abbc8a34ac8f44328596102c35e184dc => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:50.930950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-173df777-0066-4b1c-883e-9c6762981560 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:50.932176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-173df777-0066-4b1c-883e-9c6762981560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:50.998109 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1078/2145] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:41:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:41:51.059482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:51.060924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:51.162414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df777-0066-4b1c-883e-9c6762981560 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:51.163921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-173df777-0066-4b1c-883e-9c6762981560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:51.219743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be416a9e-fad3-45b7-9ada-a565a93f2651 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:51.225759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35db06-7993-45c4-be7e-e7f3fc806f1e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:51.226373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35db06-7993-45c4-be7e-e7f3fc806f1e service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.226653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35db06-7993-45c4-be7e-e7f3fc806f1e service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.226926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35db06-7993-45c4-be7e-e7f3fc806f1e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.230655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:51.238104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35db06-7993-45c4-be7e-e7f3fc806f1e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:51.239920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35db06-7993-45c4-be7e-e7f3fc806f1e service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:51.249888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1068/2146] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:41:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:41:51.250144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173df777-0066-4b1c-883e-9c6762981560 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:51.255633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:51.257411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.257411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.257411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] PATH_INFO: `/v3/projects/fa83f5942d9e44068e351f1256065ab7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.258671 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa83f5942d9e44068e351f1256065ab7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:51.260749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:51.359819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa83f5942d9e44068e351f1256065ab7', 'request_id': 'req-2703ba20-8bea-4f0d-a73a-478714588020'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:51.367841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa83f5942d9e44068e351f1256065ab7', 'request_id': 'req-2703ba20-8bea-4f0d-a73a-478714588020'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:51.377702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa83f5942d9e44068e351f1256065ab7', 'request_id': 'req-2703ba20-8bea-4f0d-a73a-478714588020'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:51.394738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] The token cache is being invalidate because project fa83f5942d9e44068e351f1256065ab7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:41:51.395059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:51.396433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2703ba20-8bea-4f0d-a73a-478714588020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:51.405533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1079/2147] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:41:51 2022] DELETE /identity/v3/projects/fa83f5942d9e44068e351f1256065ab7 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:51.410053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.410934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.411217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.436030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:51.437298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:51.472025 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:51.531839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1069/2148] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:51.540970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.541263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.541680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.565225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:51.566720 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:51.575025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1080/2149] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:51.579033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:51.590017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d4a369-660b-4783-a03a-2943aa2f43b2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:51.597572 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:51.598826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.599179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.599529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.600253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:41:51.600554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:51.602187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:51.619442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1070/2150] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:41:51.624319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:51.637965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07004d66-3025-47fe-abf6-bace3c411505 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:51.643485 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:51.644364 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.644645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.645386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.646020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:51.648138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:51.733531 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1081/2151] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:41:51 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:41:51.738535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:51.754993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7742142e-8757-4912-ab68-c2daee64b13c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:51.922307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:51.923008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:51.923193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:51.923462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:51.924091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:51.926161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:51.997763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1071/2152] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:41:51 2022] POST /identity/v3/users => generated 285 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:41:52.004179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.025194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7ce79e-f02b-4de3-aa6e-1c9f2beddd07 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.039536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.040528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.041111 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.041647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.047529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.049546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.072248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1082/2153] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:41:52.073636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.097535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4c8ad5-0a78-456d-be36-0583d528fac1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.102836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac1fcd6-85d4-4e14-aa17-064a571b6b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.103701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1fcd6-85d4-4e14-aa17-064a571b6b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.104183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1fcd6-85d4-4e14-aa17-064a571b6b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.104654 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1fcd6-85d4-4e14-aa17-064a571b6b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.105527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1fcd6-85d4-4e14-aa17-064a571b6b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.107335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac1fcd6-85d4-4e14-aa17-064a571b6b7c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.158998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.166189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1072/2154] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:41:52.172677 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a927ad-ad1c-4a50-8db7-be7077613dc1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.197514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.198529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.198873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.199214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] PATH_INFO: `/v3/projects/6949e58da64e460f8fa7c6f768b5c96f/users/a6d6a3e130f74a87bf25a4ac15fa97a3/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.222214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6949e58da64e460f8fa7c6f768b5c96f, user_id=a6d6a3e130f74a87bf25a4ac15fa97a3, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.238885 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.272928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1083/2155] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:41:52 2022] PUT /identity/v3/projects/6949e58da64e460f8fa7c6f768b5c96f/users/a6d6a3e130f74a87bf25a4ac15fa97a3/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:52.284401 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.284773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.285870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.351249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] MFA Rules not processed for user `a6d6a3e130f74a87bf25a4ac15fa97a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:52.353111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:52.463062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1073/2156] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:52.467916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.582929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fe28bd-1796-4395-b5d1-619903f4f47c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.590911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.591608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.598527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.598895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.599595 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.602484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.625726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:52.656721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1084/2157] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:41:52 2022] POST /identity/v3/projects => generated 369 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:41:52.661041 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.676567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a0efa-4b4f-456c-9598-b0e2bc8f49fe None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.709808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:52.774306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.774999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.775345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.775686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.776454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.778563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.810124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1074/2158] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:41:52 2022] POST /identity/v3/users => generated 291 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:41:52.814445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.826221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ec5b64-ecaa-4fed-9ecb-38ee00f7874b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.835608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.836493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.836829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.837407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.838204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.839919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.856222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1085/2159] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:41:52.857646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.871192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e59460-99c6-463a-896f-710d1e8e6b9d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:52.877298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:52.879393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:52.879804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:52.880139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] PATH_INFO: `/v3/projects/5096eda9bd1a42a39b23ac414837da87/users/25017f81dae242f68280b47b44698c5a/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:52.903795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5096eda9bd1a42a39b23ac414837da87, user_id=25017f81dae242f68280b47b44698c5a, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:52.907591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:52.929614 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1075/2160] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:41:52 2022] PUT /identity/v3/projects/5096eda9bd1a42a39b23ac414837da87/users/25017f81dae242f68280b47b44698c5a/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:52.931559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:52.949268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd9bbb-343e-4287-bffa-e19ddd5e34c7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:53.047529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a704f4c7-ccba-482f-ad04-d42538e63970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:53.048413 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f4c7-ccba-482f-ad04-d42538e63970 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.049128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f4c7-ccba-482f-ad04-d42538e63970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.049505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a704f4c7-ccba-482f-ad04-d42538e63970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.050226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704f4c7-ccba-482f-ad04-d42538e63970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:53.051936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a704f4c7-ccba-482f-ad04-d42538e63970 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:53.066631 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1086/2161] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:41:52 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:41:53.070398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:53.087743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cdc185-855a-499d-8823-8be8f1dc00bd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:53.092534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:53.093732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.094003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.094275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] PATH_INFO: `/v3/projects/5096eda9bd1a42a39b23ac414837da87/users/25017f81dae242f68280b47b44698c5a/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.096269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5096eda9bd1a42a39b23ac414837da87, user_id=25017f81dae242f68280b47b44698c5a, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:53.099311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:53.121599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1076/2162] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:41:53 2022] PUT /identity/v3/projects/5096eda9bd1a42a39b23ac414837da87/users/25017f81dae242f68280b47b44698c5a/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:53.123287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.123886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.124496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.168950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] MFA Rules not processed for user `25017f81dae242f68280b47b44698c5a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:53.170289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:53.285645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:53.287710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:53.288892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1087/2163] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:53 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:53.400760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45345a5f-8a68-4c21-b322-1625362236fb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:53.405104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:53.408367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.408367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.408367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.437069 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:53.438579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:53.460463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1077/2164] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:41:53.533867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:53.535525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:53.553120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065b718d-1224-4dcb-8bcd-75402115b5ef None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:53.557936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebf238-39e8-4eab-b561-37068920a1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:53.558224 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebf238-39e8-4eab-b561-37068920a1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:53.558501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebf238-39e8-4eab-b561-37068920a1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:53.558784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebf238-39e8-4eab-b561-37068920a1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:53.639319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebf238-39e8-4eab-b561-37068920a1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:53.640763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebf238-39e8-4eab-b561-37068920a1c9 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:53.663863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1088/2165] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:41:57.305699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:57.383552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6020c026-e979-4207-a560-dabfdad6e158 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:57.394644 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:57.395278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:57.395552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:57.395825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] PATH_INFO: `/v3/users/a6d6a3e130f74a87bf25a4ac15fa97a3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:57.396971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6d6a3e130f74a87bf25a4ac15fa97a3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:57.398947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:57.432165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:57.473856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6d6a3e130f74a87bf25a4ac15fa97a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:57.474549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6d6a3e130f74a87bf25a4ac15fa97a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:57.475185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6d6a3e130f74a87bf25a4ac15fa97a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:57.486085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6d6a3e130f74a87bf25a4ac15fa97a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:57.496172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6d6a3e130f74a87bf25a4ac15fa97a3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:57.519702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1078/2166] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:41:57 2022] DELETE /identity/v3/users/a6d6a3e130f74a87bf25a4ac15fa97a3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:57.524255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:57.524779 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:57.524887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:57.536490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:57.550122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:57.550721 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:57.624492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1089/2167] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:57.635175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:57.636748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:57.700735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef968517-4d60-462f-a4ee-9cd7d488117e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:57.705880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:57.706564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:57.706918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:57.707263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:57.717477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:57.719710 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:57.730066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1079/2168] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:41:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:41:58.050156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:58.053674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:58.076083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75872bb-b610-4d7b-853e-882b802cd03f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:58.084021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:58.085717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:58.085717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:58.085717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] PATH_INFO: `/v3/projects/6949e58da64e460f8fa7c6f768b5c96f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:58.087579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6949e58da64e460f8fa7c6f768b5c96f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:58.095589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:58.143323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6949e58da64e460f8fa7c6f768b5c96f', 'request_id': 'req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.149919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6949e58da64e460f8fa7c6f768b5c96f', 'request_id': 'req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.160781 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6949e58da64e460f8fa7c6f768b5c96f', 'request_id': 'req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.173828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] The token cache is being invalidate because project 6949e58da64e460f8fa7c6f768b5c96f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:41:58.174137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.174906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.176923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1090/2169] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:41:58 2022] DELETE /identity/v3/projects/6949e58da64e460f8fa7c6f768b5c96f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:58.180829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:58.190972 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad98ca58-dadf-4459-8242-98e2d99387ab service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:58.285003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:58.285823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:58.286100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:58.286372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] PATH_INFO: `/v3/users/25017f81dae242f68280b47b44698c5a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:58.287515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25017f81dae242f68280b47b44698c5a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:58.289593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:58.367949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:58.374256 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25017f81dae242f68280b47b44698c5a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.376590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25017f81dae242f68280b47b44698c5a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.376943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25017f81dae242f68280b47b44698c5a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.387776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25017f81dae242f68280b47b44698c5a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.402626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fde216a8-7815-4755-9e4d-a409c276ea61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25017f81dae242f68280b47b44698c5a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:58.428907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1080/2170] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:41:58 2022] DELETE /identity/v3/users/25017f81dae242f68280b47b44698c5a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:59.006189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.008257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:59.025593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a56c16c-2aa9-42e9-98f1-bea1a76ccdf6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.095902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.096543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.096957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.097240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] PATH_INFO: `/v3/projects/5096eda9bd1a42a39b23ac414837da87` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.100154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5096eda9bd1a42a39b23ac414837da87)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.102120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.143501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:59.159889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5096eda9bd1a42a39b23ac414837da87', 'request_id': 'req-425d8ab2-2f02-41da-980f-2ada056f100f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:59.167526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5096eda9bd1a42a39b23ac414837da87', 'request_id': 'req-425d8ab2-2f02-41da-980f-2ada056f100f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:59.179481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5096eda9bd1a42a39b23ac414837da87', 'request_id': 'req-425d8ab2-2f02-41da-980f-2ada056f100f'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:59.193875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] The token cache is being invalidate because project 5096eda9bd1a42a39b23ac414837da87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:41:59.194555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:59.195313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-425d8ab2-2f02-41da-980f-2ada056f100f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:41:59.197059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1091/2171] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:41:59 2022] DELETE /identity/v3/projects/5096eda9bd1a42a39b23ac414837da87 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:59.202536 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.203440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.203719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.226209 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:59.227520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:59.317290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1081/2172] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:59.319035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.319341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.319611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.345743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:59.347291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:59.361605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1092/2173] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:59.362355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.372699 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e2089-b103-40f9-badc-d50d6c98e8b0 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.379232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.379879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.380162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.380436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.381180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:41:59.381535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.383303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.400145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1082/2174] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:41:59.401452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.412202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870d6cc-8383-434d-9020-dc75f9121242 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.417061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.417761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.418038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.418301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.418930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.421927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.455357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1093/2175] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:41:59 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:41:59.459419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.472309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e612901e-28e8-47bd-b743-7ae00bb2c1a8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.540119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.540730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.541009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.541283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.542113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.544204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.581859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1083/2176] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:41:59 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:41:59.582076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.595689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e68e1e-b401-4e83-a008-be8b3d462a56 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.601991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.602607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.602889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.603228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.604042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.605855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.618507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1094/2177] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:41:59.622462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.645144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93fc47-fffe-49a3-bbc9-093d43e7d518 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.661417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5b607-7b24-4f39-a308-b8558b7c6baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.662281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b607-7b24-4f39-a308-b8558b7c6baa admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.662741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b607-7b24-4f39-a308-b8558b7c6baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.663186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5b607-7b24-4f39-a308-b8558b7c6baa admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.664005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b607-7b24-4f39-a308-b8558b7c6baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.665724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e5b607-7b24-4f39-a308-b8558b7c6baa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.679962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1084/2178] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:41:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:41:59.684222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.695433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeafde-b978-4656-88d8-86d45de3ad2d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.700841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.701674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.702484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.702763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] PATH_INFO: `/v3/projects/6ac316f42df845b8a4cf55fbefcc9335/users/439168be16264061aa645c412ddfc99f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.719265 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac316f42df845b8a4cf55fbefcc9335, user_id=439168be16264061aa645c412ddfc99f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.721976 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:41:59.745389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1095/2179] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:41:59 2022] PUT /identity/v3/projects/6ac316f42df845b8a4cf55fbefcc9335/users/439168be16264061aa645c412ddfc99f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:41:59.749225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d493531-af86-4e09-9640-b057bd92355a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.749548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d493531-af86-4e09-9640-b057bd92355a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.749836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d493531-af86-4e09-9640-b057bd92355a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.779606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4d493531-af86-4e09-9640-b057bd92355a None None] MFA Rules not processed for user `439168be16264061aa645c412ddfc99f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:41:59.781672 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4d493531-af86-4e09-9640-b057bd92355a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:41:59.797282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:41:59.860547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1085/2180] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:41:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:41:59.865641 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:41:59.951482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0f3907-6486-49f4-9962-9c27b0bd239a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:41:59.958268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:41:59.959395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:41:59.960239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:41:59.960727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:41:59.962017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:41:59.963923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.001341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1096/2181] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:41:59 2022] POST /identity/v3/projects => generated 365 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:42:00.004089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d493531-af86-4e09-9640-b057bd92355a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.016627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d493531-af86-4e09-9640-b057bd92355a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.084726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.085550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.085892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.086230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.087104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.089140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.122432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1086/2182] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:42:00 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:42:00.127131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.141734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab4bc4-63f6-49ac-8b33-bb1256a162f7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.146462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.147070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.147350 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.147636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.148312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.150015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.166101 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1097/2183] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:42:00.167703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.178546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400771e8-6422-4ce4-ac4f-ac41cd283574 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.183779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.184506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.184908 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.185243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] PATH_INFO: `/v3/projects/f5ac1c32090b4bd0b1f57174d2fe7d25/users/df18d6b90df64eb48f5457205287c790/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.204661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ac1c32090b4bd0b1f57174d2fe7d25, user_id=df18d6b90df64eb48f5457205287c790, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.208727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.224298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1087/2184] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:42:00 2022] PUT /identity/v3/projects/f5ac1c32090b4bd0b1f57174d2fe7d25/users/df18d6b90df64eb48f5457205287c790/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:42:00.227612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.238602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2add22-698f-4dec-9e77-0fa76f8ed5f4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.270024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:00.309874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654572d0-f436-43f7-913e-886c5f3a6c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.310825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654572d0-f436-43f7-913e-886c5f3a6c73 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.311356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654572d0-f436-43f7-913e-886c5f3a6c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.311814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654572d0-f436-43f7-913e-886c5f3a6c73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.312582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654572d0-f436-43f7-913e-886c5f3a6c73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.314211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654572d0-f436-43f7-913e-886c5f3a6c73 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.329080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1098/2185] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:42:00.330475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.342981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1c79a-f91a-43f0-85bb-b1d270f3772b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.349194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.350057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.350718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.351324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] PATH_INFO: `/v3/projects/f5ac1c32090b4bd0b1f57174d2fe7d25/users/df18d6b90df64eb48f5457205287c790/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.354070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5ac1c32090b4bd0b1f57174d2fe7d25, user_id=df18d6b90df64eb48f5457205287c790, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.359802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.371459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1088/2186] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:42:00 2022] PUT /identity/v3/projects/f5ac1c32090b4bd0b1f57174d2fe7d25/users/df18d6b90df64eb48f5457205287c790/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:00.388278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.388466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.388749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.426897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] MFA Rules not processed for user `df18d6b90df64eb48f5457205287c790`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:00.428121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:00.515057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1099/2187] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:00.530995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.533603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:00.603751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73b3f8-5f92-455e-a988-4e17ace69e74 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.608454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.609414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.609803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.610300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.620476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.622168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.637455 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1089/2188] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:42:00.778449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:00.791404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2e8f0-40ba-4605-bd36-7fa12bbd5ec7 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:00.796109 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f10619f-18f7-466b-9f4f-60b105f5b8d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:00.799387 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10619f-18f7-466b-9f4f-60b105f5b8d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:00.799855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10619f-18f7-466b-9f4f-60b105f5b8d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:00.800142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f10619f-18f7-466b-9f4f-60b105f5b8d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:00.899801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10619f-18f7-466b-9f4f-60b105f5b8d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:00.900181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f10619f-18f7-466b-9f4f-60b105f5b8d7 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:00.911741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1100/2189] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:42:05.695463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:05.711319 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:05.774103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b3da6e-fcea-460d-8e62-3e48d603df70 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:05.778635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:05.779255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:05.779533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:05.779804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] PATH_INFO: `/v3/users/439168be16264061aa645c412ddfc99f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:05.780931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] RBAC: Authorizing `identity:delete_user(user_id=439168be16264061aa645c412ddfc99f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:05.783183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:05.832611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '439168be16264061aa645c412ddfc99f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:05.833277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '439168be16264061aa645c412ddfc99f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:05.833918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '439168be16264061aa645c412ddfc99f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:05.840876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '439168be16264061aa645c412ddfc99f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:05.847799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '439168be16264061aa645c412ddfc99f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:05.867689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1090/2190] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:42:05 2022] DELETE /identity/v3/users/439168be16264061aa645c412ddfc99f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:05.872205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a340ebf-d330-4e68-ac9c-3d16f96bbf59 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:05.872778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a340ebf-d330-4e68-ac9c-3d16f96bbf59 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:05.872900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a340ebf-d330-4e68-ac9c-3d16f96bbf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:05.893259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4a340ebf-d330-4e68-ac9c-3d16f96bbf59 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:05.894517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4a340ebf-d330-4e68-ac9c-3d16f96bbf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:05.942961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:05.962892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1101/2191] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:05.972757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:05.974183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.034436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaed87-c3b6-4125-ba3a-70083178b067 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:06.038267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f9439e-b6a9-4b5a-9962-cace63a29472 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:06.039035 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9439e-b6a9-4b5a-9962-cace63a29472 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.039311 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9439e-b6a9-4b5a-9962-cace63a29472 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.039586 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f9439e-b6a9-4b5a-9962-cace63a29472 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.050221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9439e-b6a9-4b5a-9962-cace63a29472 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:06.051775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f9439e-b6a9-4b5a-9962-cace63a29472 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:06.061616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1091/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:42:06.249499 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c899a-ef32-4e3f-b64b-a2170dff3fc6 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.249814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c899a-ef32-4e3f-b64b-a2170dff3fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.250136 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c899a-ef32-4e3f-b64b-a2170dff3fc6 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.252363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1102/2193] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:42:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:42:06.263030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.263182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.263420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.319967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:06.321637 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.342581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:06.365901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c899a-ef32-4e3f-b64b-a2170dff3fc6 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:06.368124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-067c899a-ef32-4e3f-b64b-a2170dff3fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.409193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c899a-ef32-4e3f-b64b-a2170dff3fc6 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:06.413966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:06.414633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.414915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.415189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] PATH_INFO: `/v3/projects/6ac316f42df845b8a4cf55fbefcc9335` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.416338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac316f42df845b8a4cf55fbefcc9335)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:06.418343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:06.430446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1092/2194] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:06.446990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:06.448332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.467314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac316f42df845b8a4cf55fbefcc9335', 'request_id': 'req-31a8388d-ceec-4a35-aaec-cd80866bc30d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.474082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac316f42df845b8a4cf55fbefcc9335', 'request_id': 'req-31a8388d-ceec-4a35-aaec-cd80866bc30d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.481725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac316f42df845b8a4cf55fbefcc9335', 'request_id': 'req-31a8388d-ceec-4a35-aaec-cd80866bc30d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.491902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] The token cache is being invalidate because project 6ac316f42df845b8a4cf55fbefcc9335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:42:06.492186 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.492830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.494658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1103/2195] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:42:06 2022] DELETE /identity/v3/projects/6ac316f42df845b8a4cf55fbefcc9335 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:42:06.498622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:06.499961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.507012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbecd1-2db7-416f-804b-1060485a8872 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:06.557038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:06.557700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.557982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.558255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.560032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.563718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8388d-ceec-4a35-aaec-cd80866bc30d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:06.569193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:06.569821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.570462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.570462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] PATH_INFO: `/v3/users/df18d6b90df64eb48f5457205287c790` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.571522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df18d6b90df64eb48f5457205287c790)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:06.573483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:06.622223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df18d6b90df64eb48f5457205287c790'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.633104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df18d6b90df64eb48f5457205287c790'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.633774 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df18d6b90df64eb48f5457205287c790'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.634589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:06.636255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055aaed8-ca2c-48b8-87ed-8c2d4f0ed113 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:06.642182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df18d6b90df64eb48f5457205287c790'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.642842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8755256b-0474-40b6-ae18-c703c4a3f6f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df18d6b90df64eb48f5457205287c790'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:06.647713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1093/2196] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:42:06.652711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:06.665731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1104/2197] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:42:06 2022] DELETE /identity/v3/users/df18d6b90df64eb48f5457205287c790 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:42:06.722840 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488b460-8f24-4e91-9df7-5edba4bf9b96 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.723423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488b460-8f24-4e91-9df7-5edba4bf9b96 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.723423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488b460-8f24-4e91-9df7-5edba4bf9b96 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.724810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1094/2198] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:42:06.729829 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.730120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.730395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.773764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:06.775146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.835049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1105/2199] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:06.847949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488b460-8f24-4e91-9df7-5edba4bf9b96 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:06.849706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4488b460-8f24-4e91-9df7-5edba4bf9b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.863451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488b460-8f24-4e91-9df7-5edba4bf9b96 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:06.867030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dcfdc6-e2bc-46f7-a192-f7c085e7e954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:06.867631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfdc6-e2bc-46f7-a192-f7c085e7e954 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.867910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfdc6-e2bc-46f7-a192-f7c085e7e954 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.868182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dcfdc6-e2bc-46f7-a192-f7c085e7e954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.873422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:06.876698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcfdc6-e2bc-46f7-a192-f7c085e7e954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:06.879202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dcfdc6-e2bc-46f7-a192-f7c085e7e954 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:06.887441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1095/2200] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:42:06.951138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:06.952606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:06.973271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821e31a-81f8-43ef-85b5-0712e67a9c01 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.973640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821e31a-81f8-43ef-85b5-0712e67a9c01 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.973926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821e31a-81f8-43ef-85b5-0712e67a9c01 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:06.976867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1096/2201] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:42:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 12 12:42:06.982854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:06.983099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:06.983373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.005516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:07.006733 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.038880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a956b95-e94d-459b-92cb-854a4ecf7135 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.044704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.045334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.045625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.045895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] PATH_INFO: `/v3/projects/f5ac1c32090b4bd0b1f57174d2fe7d25` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.047055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5ac1c32090b4bd0b1f57174d2fe7d25)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.048963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.075100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1097/2202] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:42:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:42:07.087393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.089215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.096912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5ac1c32090b4bd0b1f57174d2fe7d25', 'request_id': 'req-246abd74-69b5-4b56-8813-b8575e69ed46'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:07.101468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0cc79e-aaf5-45be-ab7a-94dde436feb9 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.103621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5ac1c32090b4bd0b1f57174d2fe7d25', 'request_id': 'req-246abd74-69b5-4b56-8813-b8575e69ed46'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:07.111793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5ac1c32090b4bd0b1f57174d2fe7d25', 'request_id': 'req-246abd74-69b5-4b56-8813-b8575e69ed46'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:07.122559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] The token cache is being invalidate because project f5ac1c32090b4bd0b1f57174d2fe7d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:42:07.122864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:07.123562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-246abd74-69b5-4b56-8813-b8575e69ed46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:07.125908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1106/2203] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:42:06 2022] DELETE /identity/v3/projects/f5ac1c32090b4bd0b1f57174d2fe7d25 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:07.131140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a04c80-37d2-4a6f-8158-9c66753ab4ed None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.131454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a04c80-37d2-4a6f-8158-9c66753ab4ed None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.131739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a04c80-37d2-4a6f-8158-9c66753ab4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.151941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-34a04c80-37d2-4a6f-8158-9c66753ab4ed None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:07.152156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.152753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.153037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.153177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-34a04c80-37d2-4a6f-8158-9c66753ab4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.153382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.155076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.166930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:07.227967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1107/2204] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:42:07.229543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.230510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.230510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.233110 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.235573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.247265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1098/2205] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:42:07.252994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:07.254248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.269784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1108/2206] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:07.271741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.284503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d9b9e-7e77-455f-bf55-3e81dc9eb789 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.290567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.291249 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.291443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.291881 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.292647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:42:07.292992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.294570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.307570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1099/2207] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:42:07.311738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.321905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb63276-eb53-49dd-9065-a59d5f254c1d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.326272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.326878 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.327156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.327422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.328076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.330588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.365411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1109/2208] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:42:07 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:42:07.367660 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.378815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd97cd3-18d0-434f-be0c-887551c87498 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.390092 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:07.440145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.440817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.441163 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.441517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.442222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.444148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.469348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1100/2209] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:42:07 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:42:07.473291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.485141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307469c6-1fe5-4058-b179-452d2546e7fe admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.490169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.490839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.491182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.491515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.492250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.494051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.506543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1110/2210] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:42:07.510554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.521263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b0a33b-a809-45e5-83d2-a30fb409dfb2 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.526958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f8dbe-477b-4b5c-9c08-5aaec13f25fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.527913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f8dbe-477b-4b5c-9c08-5aaec13f25fc admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.528292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f8dbe-477b-4b5c-9c08-5aaec13f25fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.528568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f8dbe-477b-4b5c-9c08-5aaec13f25fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.529493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f8dbe-477b-4b5c-9c08-5aaec13f25fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.531723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56f8dbe-477b-4b5c-9c08-5aaec13f25fc admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.542952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1101/2211] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:42:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:42:07.547562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.559856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf3d87a-3306-4c5e-a2c5-8a9c4494e17a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.565982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.566691 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.566993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.573999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] PATH_INFO: `/v3/projects/96088070a322400db76bf0f74183e1dd/users/972425d739084937ba61ce0ee5cb062a/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.590337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96088070a322400db76bf0f74183e1dd, user_id=972425d739084937ba61ce0ee5cb062a, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.593150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.609426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1111/2212] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:42:07 2022] PUT /identity/v3/projects/96088070a322400db76bf0f74183e1dd/users/972425d739084937ba61ce0ee5cb062a/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:07.620587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.620691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.621105 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.651774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] MFA Rules not processed for user `972425d739084937ba61ce0ee5cb062a`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:07.652995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.723512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1102/2213] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:07.736725 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:07.738596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:07.796683 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aedbda-a7f2-4b55-b616-3945d24b6309 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:07.802074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:07.802682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:07.802963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:07.803234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:07.811914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:07.813738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:07.822673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1112/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:42:10.055771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:10.057534 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:10.142415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c4502-570d-4d00-8a04-e1e4b33d8361 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:10.148684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:10.149323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:10.149613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:10.149890 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:10.151640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:10.175478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:10.234059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:10.235670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d948a-3e05-4c21-ba5b-26cfd0370e46 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:10.253665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1103/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:42:10.856025 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:10.917418 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:10.922178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3147f864-fb39-4216-b5d2-e51624aa4443 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:10.928047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:10.928752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:10.929031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:10.929323 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] PATH_INFO: `/v3/users/972425d739084937ba61ce0ee5cb062a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:10.930447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=972425d739084937ba61ce0ee5cb062a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:10.932765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:10.980584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '972425d739084937ba61ce0ee5cb062a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:10.988748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '972425d739084937ba61ce0ee5cb062a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:10.989508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '972425d739084937ba61ce0ee5cb062a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:10.997173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '972425d739084937ba61ce0ee5cb062a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:10.998185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '972425d739084937ba61ce0ee5cb062a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:11.019922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1113/2216] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:42:10 2022] DELETE /identity/v3/users/972425d739084937ba61ce0ee5cb062a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:11.024414 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9485-9207-48c0-bf26-d4a3861f7235 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.024708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9485-9207-48c0-bf26-d4a3861f7235 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.024984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e9485-9207-48c0-bf26-d4a3861f7235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.044532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-691e9485-9207-48c0-bf26-d4a3861f7235 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:11.045792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-691e9485-9207-48c0-bf26-d4a3861f7235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.113762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1104/2217] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:11.124935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:11.126856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.193615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37a3df9-cee2-428e-b0dd-ef2e086f6aaf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:11.197181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:11.197812 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.198086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.198353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.207715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:11.209132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:11.218546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1114/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:42:11.334028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.334323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.334600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.354454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:11.355719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.416348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1105/2219] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:11.429349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:11.430689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.503085 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd39a18b-80f9-4d68-a4b1-0ff667870907 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:11.507182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:11.508759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.509576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f1d2f-f60e-47a7-a0c2-88ed50eb531c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:11.510464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1d2f-f60e-47a7-a0c2-88ed50eb531c service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.510751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1d2f-f60e-47a7-a0c2-88ed50eb531c service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.511030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f1d2f-f60e-47a7-a0c2-88ed50eb531c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.521561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f1d2f-f60e-47a7-a0c2-88ed50eb531c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:11.523177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f1d2f-f60e-47a7-a0c2-88ed50eb531c service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:11.525076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cde1a0-56af-4c0d-8ef9-dc5c10384623 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:11.530260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:11.530976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.531410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.531915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] PATH_INFO: `/v3/projects/96088070a322400db76bf0f74183e1dd` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.534040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=96088070a322400db76bf0f74183e1dd)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:11.536352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:11.539327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1115/2220] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:42:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:42:11.574306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:11.609008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96088070a322400db76bf0f74183e1dd', 'request_id': 'req-0df6141b-56da-432f-85c9-ad1ce2dcf58b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:11.622292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96088070a322400db76bf0f74183e1dd', 'request_id': 'req-0df6141b-56da-432f-85c9-ad1ce2dcf58b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:11.633863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96088070a322400db76bf0f74183e1dd', 'request_id': 'req-0df6141b-56da-432f-85c9-ad1ce2dcf58b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:11.648867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] The token cache is being invalidate because project 96088070a322400db76bf0f74183e1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:42:11.649526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:11.650388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0df6141b-56da-432f-85c9-ad1ce2dcf58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:42:11.653022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1106/2221] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:42:11 2022] DELETE /identity/v3/projects/96088070a322400db76bf0f74183e1dd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:42:11.659572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.660157 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.660763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.691476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:11.694008 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.699430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:11.816276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1116/2222] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:11.825944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.826336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.826707 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.854664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:11.856034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:11.865479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1107/2223] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:11.869559 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:11.881069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbbcd7-d413-4d95-95b7-537886d95069 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:11.886154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:11.886857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.887198 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.887563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.888342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:42:11.888741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:11.891263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:11.907166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1117/2224] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:42:11.911391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:11.921997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2fed19-34e6-4bb0-87a5-8a63ede23a8f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:11.927556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:11.928463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:11.928843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:11.929208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:11.930166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:11.932214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:11.975103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1108/2225] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:42:11 2022] POST /identity/v3/projects => generated 369 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:42:11.976071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:11.998372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a0562-eb3f-4da9-829e-66624148b29e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:12.067577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:12.068394 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:12.068777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:12.069121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:12.069868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:12.071926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:12.094644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1118/2226] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:42:11 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:42:12.098817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:12.109357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd207ec-aa40-40a8-bf39-f781a4a11dd6 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:12.114831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:12.117512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:12.118363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:12.119049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:12.120221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:12.122297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:12.135959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1109/2227] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:42:12.139933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:12.151071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341b43e9-7dbb-45e7-858c-c9e5c60e04bd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:12.157088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274537d-ec4e-4f61-bd5b-d0e8ef216a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:12.157987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274537d-ec4e-4f61-bd5b-d0e8ef216a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:12.158268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274537d-ec4e-4f61-bd5b-d0e8ef216a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:12.158544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274537d-ec4e-4f61-bd5b-d0e8ef216a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:12.159339 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274537d-ec4e-4f61-bd5b-d0e8ef216a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:12.160978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274537d-ec4e-4f61-bd5b-d0e8ef216a2c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:12.173940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1119/2228] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:42:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:42:12.177791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:12.189821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7020e9-8863-4f28-93c5-59b83246a389 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:12.194873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:12.195533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:12.195810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:12.196080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] PATH_INFO: `/v3/projects/f833f7a2a7414858ba1eead4b42a4413/users/6921bf69cdbd49b192f3b26557215e1b/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:12.212624 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] RBAC: Authorizing `identity:create_grant(project_id=f833f7a2a7414858ba1eead4b42a4413, user_id=6921bf69cdbd49b192f3b26557215e1b, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:12.215778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:12.233583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1110/2229] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:42:12 2022] PUT /identity/v3/projects/f833f7a2a7414858ba1eead4b42a4413/users/6921bf69cdbd49b192f3b26557215e1b/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:42:12.244315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957abfd4-9401-457c-8afe-02549c1e511b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:12.244681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957abfd4-9401-457c-8afe-02549c1e511b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:12.245021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957abfd4-9401-457c-8afe-02549c1e511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:12.278377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-957abfd4-9401-457c-8afe-02549c1e511b None None] MFA Rules not processed for user `6921bf69cdbd49b192f3b26557215e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:42:12.279606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-957abfd4-9401-457c-8afe-02549c1e511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:12.370704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1120/2230] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:42:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:42:12.384095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:42:12.386535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:42:12.470225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1271ebbf-3dc8-49c3-985a-9d39b6decbde admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:42:12.473965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:42:12.474579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:42:12.474863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:42:12.475126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:42:12.484048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:42:12.486289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:42:12.496267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1111/2231] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:42:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:42:22.359664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:42:22.491511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:01.762058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053cd29-b63b-4c49-986e-964935663226 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:01.762600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053cd29-b63b-4c49-986e-964935663226 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:01.762715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053cd29-b63b-4c49-986e-964935663226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:01.782192 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2053cd29-b63b-4c49-986e-964935663226 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:01.783391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2053cd29-b63b-4c49-986e-964935663226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:01.838756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1121/2232] 10.222.0.17 () {58 vars in 967 bytes} [Wed Oct 12 12:43:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:01.861155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:01.862595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:01.937635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bcdcd-63ce-4a93-bf4e-3f74082a9e54 service glance] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:01.943393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1c40f7-6110-4a51-bbe5-22c66e20f67c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:01.944000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c40f7-6110-4a51-bbe5-22c66e20f67c service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:01.944281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c40f7-6110-4a51-bbe5-22c66e20f67c service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:01.944575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c40f7-6110-4a51-bbe5-22c66e20f67c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:01.953086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c40f7-6110-4a51-bbe5-22c66e20f67c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:01.954698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1c40f7-6110-4a51-bbe5-22c66e20f67c service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:01.963697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1112/2233] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:43:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:43:04.371563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053cd29-b63b-4c49-986e-964935663226 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:04.435255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053cd29-b63b-4c49-986e-964935663226 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:04.439603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:04.440373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:04.440647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:04.440927 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] PATH_INFO: `/v3/users/6921bf69cdbd49b192f3b26557215e1b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:04.442206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6921bf69cdbd49b192f3b26557215e1b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:04.444982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:04.493787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6921bf69cdbd49b192f3b26557215e1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:04.501644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6921bf69cdbd49b192f3b26557215e1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:04.502419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6921bf69cdbd49b192f3b26557215e1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:04.510045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6921bf69cdbd49b192f3b26557215e1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:04.510680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6921bf69cdbd49b192f3b26557215e1b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:04.527970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1122/2234] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:04 2022] DELETE /identity/v3/users/6921bf69cdbd49b192f3b26557215e1b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:04.532449 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:04.532907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:04.533031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:04.553297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:04.554800 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:04.630079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1113/2235] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:04.644849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:04.648213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:04.677169 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:04.713349 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051ad63-490e-4b0a-8606-3ed2304ec077 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:04.717042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb3d76b-1000-46ec-b60f-595cecbf9aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:04.717786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d76b-1000-46ec-b60f-595cecbf9aba service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:04.718131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d76b-1000-46ec-b60f-595cecbf9aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:04.718466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb3d76b-1000-46ec-b60f-595cecbf9aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:04.729173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3d76b-1000-46ec-b60f-595cecbf9aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:04.730727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb3d76b-1000-46ec-b60f-595cecbf9aba service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:04.741151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1123/2236] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:05.037966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.039457 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:05.057933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08088f4b-1863-4119-8a2b-df553f50d02e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.071623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.072268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.072570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.072825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] PATH_INFO: `/v3/projects/f833f7a2a7414858ba1eead4b42a4413` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.073987 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f833f7a2a7414858ba1eead4b42a4413)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.075943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.124708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f833f7a2a7414858ba1eead4b42a4413', 'request_id': 'req-f52687aa-9bb9-44c0-ba66-08250f4f6c72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:05.134441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f833f7a2a7414858ba1eead4b42a4413', 'request_id': 'req-f52687aa-9bb9-44c0-ba66-08250f4f6c72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:05.141155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f833f7a2a7414858ba1eead4b42a4413', 'request_id': 'req-f52687aa-9bb9-44c0-ba66-08250f4f6c72'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:05.152574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] The token cache is being invalidate because project f833f7a2a7414858ba1eead4b42a4413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:05.152870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:05.153606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f52687aa-9bb9-44c0-ba66-08250f4f6c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:05.155255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1114/2237] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:05 2022] DELETE /identity/v3/projects/f833f7a2a7414858ba1eead4b42a4413 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:05.160646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.161122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.161504 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.181620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:05.182845 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:05.246092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1124/2238] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:05.253905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.254187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.254792 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.290649 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:05.292304 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:05.300498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1115/2239] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:05.305235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.315278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cfcb2-0e17-41a2-9cb3-8eadea1228c0 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.320360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.321076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.321389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.321665 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.322312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:05.322608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.324202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.340029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1125/2240] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:05.341097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.350592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eeeac0-39c2-4c6c-ade6-c94d180460cd None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.355331 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.355930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.356210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.356483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.357124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.359077 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.363627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:05.393530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1116/2241] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:05 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:05.394628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.405451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a6e4b-e30c-485e-a544-50ad87f0c013 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.462742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.463361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.463645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.463920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.464608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.466596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.489058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1126/2242] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:05 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:05.493747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.504573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45488c65-3564-4cdf-8b89-554e3f924b56 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.513980 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.514931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.515214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.515479 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.516437 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.517763 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.532156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1117/2243] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:05.533579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.544204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1571e9-0ecc-4c06-9808-5be9bfee6142 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.549114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101081ac-916e-45cd-b16c-c6a21fb1bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.549612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101081ac-916e-45cd-b16c-c6a21fb1bf71 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.549888 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101081ac-916e-45cd-b16c-c6a21fb1bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.550161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101081ac-916e-45cd-b16c-c6a21fb1bf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.550803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101081ac-916e-45cd-b16c-c6a21fb1bf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.552416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101081ac-916e-45cd-b16c-c6a21fb1bf71 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.564661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1127/2244] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:05.569574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.579799 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49131812-34ce-4213-bd50-6835878bd3df admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.587754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.589063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.589513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.589907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] PATH_INFO: `/v3/projects/5e79b1a39737453bb5a7de18ff0df02d/users/714db08714f1469ea46c1bb6a7702c29/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.616562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e79b1a39737453bb5a7de18ff0df02d, user_id=714db08714f1469ea46c1bb6a7702c29, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.621893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.642599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1118/2245] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:05 2022] PUT /identity/v3/projects/5e79b1a39737453bb5a7de18ff0df02d/users/714db08714f1469ea46c1bb6a7702c29/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:05.654460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.654793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.655053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.708021 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] MFA Rules not processed for user `714db08714f1469ea46c1bb6a7702c29`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:05.711821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:05.752773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:05.838529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1128/2246] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:05.845048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:05.846766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:05.912979 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445552b2-dfe6-4335-ac44-0b169253a106 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:05.916846 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a725fb-944c-498c-b8af-08744e3c6f08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:05.917559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a725fb-944c-498c-b8af-08744e3c6f08 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:05.917911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a725fb-944c-498c-b8af-08744e3c6f08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:05.918259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a725fb-944c-498c-b8af-08744e3c6f08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:05.927600 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a725fb-944c-498c-b8af-08744e3c6f08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:05.929725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a725fb-944c-498c-b8af-08744e3c6f08 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:05.940845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1119/2247] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:05.971981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:06.035492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7086cc-94e9-4ecc-9d56-5ee5bcb7baa2 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:06.040505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:06.041140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.041430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.041700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] PATH_INFO: `/v3/users/714db08714f1469ea46c1bb6a7702c29` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.042831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=714db08714f1469ea46c1bb6a7702c29)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:06.044766 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:06.091272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '714db08714f1469ea46c1bb6a7702c29'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.100232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '714db08714f1469ea46c1bb6a7702c29'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.100984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '714db08714f1469ea46c1bb6a7702c29'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.108207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '714db08714f1469ea46c1bb6a7702c29'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.108924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '714db08714f1469ea46c1bb6a7702c29'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.126907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1129/2248] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:05 2022] DELETE /identity/v3/users/714db08714f1469ea46c1bb6a7702c29 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:06.131662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.131947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.132222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.151693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:06.152920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:06.220933 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1120/2249] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:06.231663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:06.233074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:06.303842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:06.323210 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc196-1e07-4b39-91cb-ee6492c8bcc5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:06.327452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e429e-4899-42ff-bf90-417b38f56387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:06.328050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e429e-4899-42ff-bf90-417b38f56387 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.328378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e429e-4899-42ff-bf90-417b38f56387 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.328722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e429e-4899-42ff-bf90-417b38f56387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.339012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e429e-4899-42ff-bf90-417b38f56387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:06.340550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e429e-4899-42ff-bf90-417b38f56387 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:06.352574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1130/2250] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:06.656065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:06.659391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:06.679647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9f778-9fd2-4892-ba29-f9a01b34a374 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:06.685548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:06.686138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.686424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.686677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] PATH_INFO: `/v3/projects/5e79b1a39737453bb5a7de18ff0df02d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.687805 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e79b1a39737453bb5a7de18ff0df02d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:06.689862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:06.736245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:06.737831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e79b1a39737453bb5a7de18ff0df02d', 'request_id': 'req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.746327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e79b1a39737453bb5a7de18ff0df02d', 'request_id': 'req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.753645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e79b1a39737453bb5a7de18ff0df02d', 'request_id': 'req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.764565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] The token cache is being invalidate because project 5e79b1a39737453bb5a7de18ff0df02d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:06.764857 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.765512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-0f6ce9db-23e6-4151-b0e4-64ef7c240c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:06.767008 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1121/2251] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:06 2022] DELETE /identity/v3/projects/5e79b1a39737453bb5a7de18ff0df02d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:06.772365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.772879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.772984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.792813 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:06.794051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:06.860087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1131/2252] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:06.869773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.870184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.870551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.891743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:06.892951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:06.901852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1122/2253] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:06.906225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:06.916552 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbe8df-5c1d-4868-872b-ec7b3532e092 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:06.921373 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:06.921978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.922252 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.922521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.923160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:06.923459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:06.925051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:06.938471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1132/2254] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:06.942870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:06.952120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00acb530-6e66-4926-8f6f-4d170054aa74 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:06.956947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:06.957856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:06.957856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:06.958139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:06.959312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:06.961229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:06.991134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1123/2255] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:06 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:06.995533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:07.007160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7086df-212c-492c-93b1-2860bab9f0c9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:07.094704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:07.095376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:07.095659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:07.095938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:07.096576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:07.098543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:07.121592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1133/2256] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:06 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:07.126226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:07.137047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc40d5e5-b126-4ffe-9068-71119a0c179b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:07.141682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:07.142278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:07.142561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:07.142831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:07.143616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:07.145231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:07.155817 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1124/2257] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:07.159790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:07.170484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f36eab-0298-4d98-8474-3cf7adff4190 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:07.175190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248e72c-7a44-47ab-9ee8-6944402803b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:07.175790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248e72c-7a44-47ab-9ee8-6944402803b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:07.176072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248e72c-7a44-47ab-9ee8-6944402803b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:07.176343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248e72c-7a44-47ab-9ee8-6944402803b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:07.176978 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248e72c-7a44-47ab-9ee8-6944402803b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:07.178647 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248e72c-7a44-47ab-9ee8-6944402803b5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:07.188583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1134/2258] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:07.192725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:07.202658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d624b-099e-4362-8cbd-e18be5a2f8ae admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:07.207273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:07.207911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:07.208190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:07.208463 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] PATH_INFO: `/v3/projects/26b999acd52445db85066f3c23644118/users/c7323d37023d49a1a4692fe17593dea2/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:07.223568 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=26b999acd52445db85066f3c23644118, user_id=c7323d37023d49a1a4692fe17593dea2, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:07.228969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:07.242944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1125/2259] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:07 2022] PUT /identity/v3/projects/26b999acd52445db85066f3c23644118/users/c7323d37023d49a1a4692fe17593dea2/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:07.253623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:07.253940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:07.254218 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:07.287967 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] MFA Rules not processed for user `c7323d37023d49a1a4692fe17593dea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:07.289184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:07.330946 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:07.358667 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1135/2260] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:07.370199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:07.371931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:07.431370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3548-2bae-4394-8a11-bbb303da6bfd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:07.438328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:07.439023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:07.439317 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:07.439578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:07.449280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:07.450832 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:07.459998 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1126/2261] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:15.434970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:15.436360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:15.497118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306362d5-849d-4238-9a51-98f36a865f32 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:15.500910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:15.501543 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:15.501824 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:15.502104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:15.503826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:15.573150 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:15.574625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547f75-482e-406e-9204-f3701b8e594b service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:15.584510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1136/2262] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:16.562247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:16.574887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d06db-bdc8-4a2a-a8d7-2e282e175fe5 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:16.580011 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:16.580734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:16.581083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:16.581511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] PATH_INFO: `/v3/users/c7323d37023d49a1a4692fe17593dea2` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:16.584326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7323d37023d49a1a4692fe17593dea2)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:16.586969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:16.645533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7323d37023d49a1a4692fe17593dea2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:16.646697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7323d37023d49a1a4692fe17593dea2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:16.647404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7323d37023d49a1a4692fe17593dea2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:16.655117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7323d37023d49a1a4692fe17593dea2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:16.665524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7323d37023d49a1a4692fe17593dea2'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:16.673602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=84,keystone:INSERT=4,keystone:DELETE=12 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:16.691154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1127/2263] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:16 2022] DELETE /identity/v3/users/c7323d37023d49a1a4692fe17593dea2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:16.696242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:16.696830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:16.696830 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:16.721816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:16.723270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:16.791828 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1137/2264] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:16.802985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:16.804413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:16.878272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8880-373b-455d-a304-900e32bfa42d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:16.884326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:16.885408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:16.885932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:16.886404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:16.898151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:16.899508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:16.910436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1128/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:43:17.444746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:17.446576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:17.464467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5f0ef-338b-483a-a275-04711bd53535 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:17.471326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:17.472411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:17.472855 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:17.473237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] PATH_INFO: `/v3/projects/26b999acd52445db85066f3c23644118` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:17.476251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] RBAC: Authorizing `identity:delete_project(project_id=26b999acd52445db85066f3c23644118)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:17.479357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:17.527864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:17.557955 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26b999acd52445db85066f3c23644118', 'request_id': 'req-ee2b1828-79f3-41f3-b5ca-7458d56feace'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:17.566979 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26b999acd52445db85066f3c23644118', 'request_id': 'req-ee2b1828-79f3-41f3-b5ca-7458d56feace'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:17.577435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26b999acd52445db85066f3c23644118', 'request_id': 'req-ee2b1828-79f3-41f3-b5ca-7458d56feace'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:17.593703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] The token cache is being invalidate because project 26b999acd52445db85066f3c23644118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:17.594623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:17.595287 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:17.597072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1138/2266] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:17 2022] DELETE /identity/v3/projects/26b999acd52445db85066f3c23644118 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:18.690452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:18.695297 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:18.761560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:18.765131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:18.812754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0c5d-165f-4ca2-a15b-682a5ca05ff4 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:18.845965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2b1828-79f3-41f3-b5ca-7458d56feace admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:18.852239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:18.852865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:18.853137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:18.853441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] PATH_INFO: `/v3/users/383004527ce84e4f9fc40760a040936e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:18.854578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=383004527ce84e4f9fc40760a040936e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:18.856526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:18.873527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:18.874126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:18.874405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:18.874675 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:18.876646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:18.888966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:18.890602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:18.901277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1129/2267] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:18.906488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '383004527ce84e4f9fc40760a040936e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:18.914652 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '383004527ce84e4f9fc40760a040936e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:18.915436 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '383004527ce84e4f9fc40760a040936e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:18.924809 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '383004527ce84e4f9fc40760a040936e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:18.925551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-7efb65f5-6255-4fdb-bd30-dd98f857eef4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '383004527ce84e4f9fc40760a040936e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:18.944751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1139/2268] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:18 2022] DELETE /identity/v3/users/383004527ce84e4f9fc40760a040936e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:19.118594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.129843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85ebaa1-5512-448b-a768-cb57b1fae922 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.150173 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:19.189452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.190075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.190380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.190650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] PATH_INFO: `/v3/projects/557de755e1e8423f9ba4e1be0627ca53` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.191768 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] RBAC: Authorizing `identity:delete_project(project_id=557de755e1e8423f9ba4e1be0627ca53)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.194495 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.240007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '557de755e1e8423f9ba4e1be0627ca53', 'request_id': 'req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:19.247229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '557de755e1e8423f9ba4e1be0627ca53', 'request_id': 'req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:19.253570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '557de755e1e8423f9ba4e1be0627ca53', 'request_id': 'req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:19.268336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] The token cache is being invalidate because project 557de755e1e8423f9ba4e1be0627ca53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:19.268628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:19.269328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-f05afdf7-8e6c-4333-84e1-eabcc6c2328e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:19.271155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1130/2269] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:19 2022] DELETE /identity/v3/projects/557de755e1e8423f9ba4e1be0627ca53 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:19.276361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.276715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.276993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.296605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:19.297831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:19.326052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:19.358138 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1140/2270] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:19.366893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.367172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.367443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.388565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:19.389780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:19.397875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1131/2271] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:19.402095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.412388 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d319ee0-f290-407f-a6cd-c29d01409487 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.416984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.417654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.417937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.418214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.418865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:19.419178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.420836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.433212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1141/2272] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:19.437361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.446411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cb723b-116d-4e1a-a85a-1c0f02de6385 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.450945 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.451524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.451795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.452060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.452686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.454595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.480875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1132/2273] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:19.485202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.495738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976bd0-200c-4c3d-b3ee-8a1c5561dbc6 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.551574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.552685 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.552990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.553222 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.553869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.555764 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.575624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1142/2274] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:19.579615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.590764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb05d1-583c-4fce-b78c-05c1e2c86ca4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.595703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.596291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.596567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.596835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.597508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.600037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.610734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1133/2275] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:19.615490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.625540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfd0e81-d273-42c2-98d6-c107d7992928 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.630474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0faa8-6041-47ae-9693-6bdab6ef93dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.631146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0faa8-6041-47ae-9693-6bdab6ef93dd admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.631428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0faa8-6041-47ae-9693-6bdab6ef93dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.631728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0faa8-6041-47ae-9693-6bdab6ef93dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.632367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0faa8-6041-47ae-9693-6bdab6ef93dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.634066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac0faa8-6041-47ae-9693-6bdab6ef93dd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.644511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1143/2276] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:43:19.648898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.658929 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f547fad-9fb9-4f58-8a30-9b0ab3cee114 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.664112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.664741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.665015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.665283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] PATH_INFO: `/v3/projects/79e81ed4b6564ea19528c28b6ef3be6a/users/872e66829337418986e0a84daa16228c/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.680443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=79e81ed4b6564ea19528c28b6ef3be6a, user_id=872e66829337418986e0a84daa16228c, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.683450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.696867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1134/2277] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:19 2022] PUT /identity/v3/projects/79e81ed4b6564ea19528c28b6ef3be6a/users/872e66829337418986e0a84daa16228c/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:19.708538 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.708937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.709438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.736804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] MFA Rules not processed for user `872e66829337418986e0a84daa16228c`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:19.738049 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:19.801106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1144/2278] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:19.806120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.854322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:19.868736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f3a512-d513-4a27-9da0-15caeef85acb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.873276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.873900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.874175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.874442 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.875072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.877208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:19.903478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1135/2279] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:19.907735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:19.919178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c847e7-8c50-4e64-98d8-29c5487d4a76 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:19.944556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:19.976241 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:19.976843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:19.977124 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:19.977440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:19.978079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:19.979995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.000556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1145/2280] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:19 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:20.004548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.014703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2105469-197b-4a9f-a2eb-93d98c5aef6b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.019562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.020153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.020427 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.021234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.021898 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.023474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.034148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1136/2281] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:20.038637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.048733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d03c0-f5e0-43b3-8bbd-e23db7a2e9b8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.053933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9411da-1a1f-4095-9c72-1089e5ca3846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.054526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9411da-1a1f-4095-9c72-1089e5ca3846 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.054804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9411da-1a1f-4095-9c72-1089e5ca3846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.055074 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9411da-1a1f-4095-9c72-1089e5ca3846 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.055702 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9411da-1a1f-4095-9c72-1089e5ca3846 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.057369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9411da-1a1f-4095-9c72-1089e5ca3846 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.067030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1146/2282] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:20.071650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.081532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58891c3-8f41-4791-bc3a-33b982c4984b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.086682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.087321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.087602 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.087900 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] PATH_INFO: `/v3/projects/6b7603402cbe4f34af048cf0c5463900/users/4dcf427a20634feba9fc39f3f031315a/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.102425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7603402cbe4f34af048cf0c5463900, user_id=4dcf427a20634feba9fc39f3f031315a, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.105073 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.117911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1137/2283] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:20 2022] PUT /identity/v3/projects/6b7603402cbe4f34af048cf0c5463900/users/4dcf427a20634feba9fc39f3f031315a/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:20.128274 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.128717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.129056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.160113 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] MFA Rules not processed for user `4dcf427a20634feba9fc39f3f031315a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:20.161356 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:20.228904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1147/2284] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:20.233445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.296302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d917a1cc-5fd5-44c6-9758-15d0e76aa521 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.301225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.301896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.302176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.302447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.303532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.305808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.332687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1138/2285] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:20 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:20.337331 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.349091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c4940-41b1-49a2-897f-f5111a92116e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.404063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.404656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.404929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.405197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.405854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.407762 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.428735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1148/2286] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:20 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:20.432868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.443520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a855e77-cbde-4231-8ddb-e798e26ab508 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.448076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.448661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.448936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.449203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.449892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.451475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.461875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1139/2287] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:43:20.465898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.476115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954fd791-45a8-48d1-9439-bdc875f7c730 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.480696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.481366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.481650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.481915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] PATH_INFO: `/v3/projects/67fc14a7381f43598666f0e21bbfe1ef/users/325d0ce3b53847f4b447cda2818c7bb8/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.497104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67fc14a7381f43598666f0e21bbfe1ef, user_id=325d0ce3b53847f4b447cda2818c7bb8, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.499797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.513383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1149/2288] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:20 2022] PUT /identity/v3/projects/67fc14a7381f43598666f0e21bbfe1ef/users/325d0ce3b53847f4b447cda2818c7bb8/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:20.517373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.527012 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109c319-350c-4338-8200-b269af1f81a0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.581524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a900c48-b80d-4bf6-9d91-d98ce0bfa1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.582114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a900c48-b80d-4bf6-9d91-d98ce0bfa1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.582388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a900c48-b80d-4bf6-9d91-d98ce0bfa1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.582657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a900c48-b80d-4bf6-9d91-d98ce0bfa1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.583544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a900c48-b80d-4bf6-9d91-d98ce0bfa1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.585072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a900c48-b80d-4bf6-9d91-d98ce0bfa1a7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.594725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1140/2289] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:43:20.598932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.604511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:20.610739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fe381-deb8-48cd-beb8-e6446b58fbb0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.616477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.617121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.617435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.617712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] PATH_INFO: `/v3/projects/67fc14a7381f43598666f0e21bbfe1ef/users/325d0ce3b53847f4b447cda2818c7bb8/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.619633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67fc14a7381f43598666f0e21bbfe1ef, user_id=325d0ce3b53847f4b447cda2818c7bb8, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.622380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.633578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1150/2290] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:20 2022] PUT /identity/v3/projects/67fc14a7381f43598666f0e21bbfe1ef/users/325d0ce3b53847f4b447cda2818c7bb8/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:20.643814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.644108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.644391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.664973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:20.675526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] MFA Rules not processed for user `325d0ce3b53847f4b447cda2818c7bb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:20.677292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:20.744205 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1141/2291] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:20 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:20.755836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:20.757581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:20.814060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd03db8-9e07-40c2-879e-6f5b0f61a2a8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:20.817704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:20.818477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:20.818757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:20.819035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:20.828035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:20.829501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:20.838975 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1151/2292] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:20 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:21.334485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:21.335887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:21.348395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0482ae-900a-4053-a14b-f73a601fa92a None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:21.352913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:21.353544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:21.353845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:21.354102 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:21.409783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:21.411834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:21.420652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1142/2293] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:25.429609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:25.438172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a57ebc-a2f3-416c-abf6-8374673d60ee service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:25.441817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73ef1c-0523-49e1-9883-ff88935cc6b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:25.442407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ef1c-0523-49e1-9883-ff88935cc6b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:25.442693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ef1c-0523-49e1-9883-ff88935cc6b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:25.443035 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73ef1c-0523-49e1-9883-ff88935cc6b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:25.498585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ef1c-0523-49e1-9883-ff88935cc6b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:25.500283 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73ef1c-0523-49e1-9883-ff88935cc6b9 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:25.508997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1152/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:25 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:27.590108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:27.653510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1218df6-0c17-4d11-9442-a5f29316555e service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:27.658016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:27.658630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:27.658905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:27.659175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] PATH_INFO: `/v3/users/872e66829337418986e0a84daa16228c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:27.660301 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=872e66829337418986e0a84daa16228c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:27.662773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:27.700528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:27.711044 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872e66829337418986e0a84daa16228c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:27.711688 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872e66829337418986e0a84daa16228c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:27.712283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872e66829337418986e0a84daa16228c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:27.718968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872e66829337418986e0a84daa16228c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:27.726593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872e66829337418986e0a84daa16228c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:27.744273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1143/2295] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:27 2022] DELETE /identity/v3/users/872e66829337418986e0a84daa16228c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:27.748743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:27.749223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:27.749359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:27.769410 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:27.770613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:27.828179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1153/2296] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:27.837964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:27.839345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:27.896918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14acf4-45ac-4d33-a70d-610cfaf3bbbd admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:27.900410 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:27.901006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:27.901283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:27.901580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:27.910517 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:27.912713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:27.922049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1144/2297] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:28.183401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:28.184964 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:28.196931 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f67977-268a-4d34-8430-72e7c86ad451 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:28.201590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:28.202223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:28.202515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:28.202770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] PATH_INFO: `/v3/projects/79e81ed4b6564ea19528c28b6ef3be6a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:28.203900 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79e81ed4b6564ea19528c28b6ef3be6a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:28.205875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:28.237507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:28.256906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79e81ed4b6564ea19528c28b6ef3be6a', 'request_id': 'req-8ee26f12-617f-4652-92b7-f8ae973eaa16'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.263707 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79e81ed4b6564ea19528c28b6ef3be6a', 'request_id': 'req-8ee26f12-617f-4652-92b7-f8ae973eaa16'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.273179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79e81ed4b6564ea19528c28b6ef3be6a', 'request_id': 'req-8ee26f12-617f-4652-92b7-f8ae973eaa16'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.284280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] The token cache is being invalidate because project 79e81ed4b6564ea19528c28b6ef3be6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:28.284566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.285209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.286736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1154/2298] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:28 2022] DELETE /identity/v3/projects/79e81ed4b6564ea19528c28b6ef3be6a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:28.290922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:28.300499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070194d8-c538-4241-a9e6-d5d7d0af046b service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:28.355693 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:28.356309 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:28.356587 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:28.356860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] PATH_INFO: `/v3/users/4dcf427a20634feba9fc39f3f031315a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:28.357982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dcf427a20634feba9fc39f3f031315a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:28.359904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:28.405645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dcf427a20634feba9fc39f3f031315a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.406279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dcf427a20634feba9fc39f3f031315a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.406870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dcf427a20634feba9fc39f3f031315a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.414405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dcf427a20634feba9fc39f3f031315a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.416014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:28.422160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dcf427a20634feba9fc39f3f031315a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.440519 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1145/2299] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:28 2022] DELETE /identity/v3/users/4dcf427a20634feba9fc39f3f031315a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:28.719158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:28.720680 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:28.734660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee26f12-617f-4652-92b7-f8ae973eaa16 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:28.796314 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:28.797009 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:28.797271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:28.797585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] PATH_INFO: `/v3/projects/6b7603402cbe4f34af048cf0c5463900` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:28.798871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b7603402cbe4f34af048cf0c5463900)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:28.800850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:28.848713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7603402cbe4f34af048cf0c5463900', 'request_id': 'req-5fd1640c-3fd6-4df4-8fce-56591d25b98d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.855036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7603402cbe4f34af048cf0c5463900', 'request_id': 'req-5fd1640c-3fd6-4df4-8fce-56591d25b98d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.862033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7603402cbe4f34af048cf0c5463900', 'request_id': 'req-5fd1640c-3fd6-4df4-8fce-56591d25b98d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.872921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] The token cache is being invalidate because project 6b7603402cbe4f34af048cf0c5463900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:28.873205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.873881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:28.875623 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1155/2300] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:28 2022] DELETE /identity/v3/projects/6b7603402cbe4f34af048cf0c5463900 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:28.879810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:28.882098 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:28.895039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a92f4-ce3d-4e6d-a2ce-66d5cc2b1848 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:28.951553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:28.952172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:28.952441 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:28.952717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] PATH_INFO: `/v3/users/325d0ce3b53847f4b447cda2818c7bb8` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:28.953847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] RBAC: Authorizing `identity:delete_user(user_id=325d0ce3b53847f4b447cda2818c7bb8)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:28.955756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.000591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '325d0ce3b53847f4b447cda2818c7bb8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.001268 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '325d0ce3b53847f4b447cda2818c7bb8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.001902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '325d0ce3b53847f4b447cda2818c7bb8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.009411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '325d0ce3b53847f4b447cda2818c7bb8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.017645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-75e4a6d3-a4e7-4a93-b96e-7104433ff53a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '325d0ce3b53847f4b447cda2818c7bb8'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.034283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1146/2301] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:28 2022] DELETE /identity/v3/users/325d0ce3b53847f4b447cda2818c7bb8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:29.301768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.304044 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:29.322412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1640c-3fd6-4df4-8fce-56591d25b98d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.364284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:29.380337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.381058 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.381362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.381646 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] PATH_INFO: `/v3/projects/67fc14a7381f43598666f0e21bbfe1ef` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.383205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=67fc14a7381f43598666f0e21bbfe1ef)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.385122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.431424 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67fc14a7381f43598666f0e21bbfe1ef', 'request_id': 'req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.437464 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67fc14a7381f43598666f0e21bbfe1ef', 'request_id': 'req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.445125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67fc14a7381f43598666f0e21bbfe1ef', 'request_id': 'req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.455778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] The token cache is being invalidate because project 67fc14a7381f43598666f0e21bbfe1ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:29.456066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.456710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-9b8f5d36-768c-42a3-aace-ef1ede14bf8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:29.458201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1156/2302] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:29 2022] DELETE /identity/v3/projects/67fc14a7381f43598666f0e21bbfe1ef => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:29.464865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.465148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.465464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.485264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:29.486498 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:29.532757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:29.546694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1147/2303] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:29.555361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.555578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.555843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.575064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:29.576247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:29.590073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1157/2304] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:29.594369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.604794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53871026-de07-422e-9e3a-9c3712b3845c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.609252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.609875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.610151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.610416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.611048 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:29.611346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.612932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.624473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1148/2305] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:43:29.628759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.637838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b895a3-4293-498c-8fea-63972971fe54 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.642381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.642959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.643233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.643502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.644121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.646613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.673741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1158/2306] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:29 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:29.677833 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.687852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96756414-d493-4787-baf6-89bae2618a09 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.742496 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.743083 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.743359 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.743630 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.744253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.746186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.767308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1149/2307] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:29 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:29.772284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.782579 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b92b53-d49d-4505-822f-ac5c0f0f1cd1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.787490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.788079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.788357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.788627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.789259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.790892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.800622 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1159/2308] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:29.804618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.814715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46475158-c7e0-478b-8ec0-e9696dff2e70 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.819888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761b2ea-a9c3-4d74-8f02-6154a7d8237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.820476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761b2ea-a9c3-4d74-8f02-6154a7d8237c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.820759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761b2ea-a9c3-4d74-8f02-6154a7d8237c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.821033 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761b2ea-a9c3-4d74-8f02-6154a7d8237c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.821711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761b2ea-a9c3-4d74-8f02-6154a7d8237c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.823288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761b2ea-a9c3-4d74-8f02-6154a7d8237c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.834461 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1150/2309] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:29.838669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:29.848469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fdf2af-301e-489f-80ef-c66c2fece0fe admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:29.853088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:29.853758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.854042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.854315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] PATH_INFO: `/v3/projects/e4ffab40870d44fc9a63dab203c0b19b/users/ae3c586feea24c04949956cc00ce2dfc/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.868709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4ffab40870d44fc9a63dab203c0b19b, user_id=ae3c586feea24c04949956cc00ce2dfc, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:29.871391 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:29.885290 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1160/2310] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:29 2022] PUT /identity/v3/projects/e4ffab40870d44fc9a63dab203c0b19b/users/ae3c586feea24c04949956cc00ce2dfc/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:29.896439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:29.897865 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:29.898151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:29.925494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] MFA Rules not processed for user `ae3c586feea24c04949956cc00ce2dfc`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:29.926784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:29.990379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1151/2311] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:29.994863 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.055344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ee5759-01b8-4364-9033-e750c11f569d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.059930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.060519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.060798 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.061070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.061716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.063613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.078820 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:30.091999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1161/2312] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:29 2022] POST /identity/v3/projects => generated 399 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:30.096140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.106783 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0f6c2b-ffdd-46c2-889f-c118f3907743 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.151565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:30.163721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.164323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.164603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.164904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.165557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.167470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.188746 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1152/2313] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:30 2022] POST /identity/v3/users => generated 306 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:30.193478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.203799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf0f7-9c04-477c-a5c3-247463e52f2c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.208493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.209097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.209407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.209687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.210327 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.211930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.222280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1162/2314] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:30.226469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.236404 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0debf-89d3-4d2b-abef-a3fa5bffd0ee admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.241530 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.242172 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.242451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.242726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] PATH_INFO: `/v3/projects/f333b21cf0a44c88a8ee5703c4946fbf/users/e51099834f564c8aa4df9bd11e848a4a/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.258137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=f333b21cf0a44c88a8ee5703c4946fbf, user_id=e51099834f564c8aa4df9bd11e848a4a, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.260810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.274434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1153/2315] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:30 2022] PUT /identity/v3/projects/f333b21cf0a44c88a8ee5703c4946fbf/users/e51099834f564c8aa4df9bd11e848a4a/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:30.278468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.288676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75564fe-6038-4d31-acb7-9d7c059e3263 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.345872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99395de-927f-41ae-b0ea-7db3a5711194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.346404 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99395de-927f-41ae-b0ea-7db3a5711194 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.346684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99395de-927f-41ae-b0ea-7db3a5711194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.346957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99395de-927f-41ae-b0ea-7db3a5711194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.347591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99395de-927f-41ae-b0ea-7db3a5711194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.349240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99395de-927f-41ae-b0ea-7db3a5711194 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.358832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1163/2316] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:30 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:30.363064 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.373770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99e2671-9b3a-4163-bfab-feb69e5561ac admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.378780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.379412 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.379687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.379952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] PATH_INFO: `/v3/projects/f333b21cf0a44c88a8ee5703c4946fbf/users/e51099834f564c8aa4df9bd11e848a4a/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.383060 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f333b21cf0a44c88a8ee5703c4946fbf, user_id=e51099834f564c8aa4df9bd11e848a4a, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.385755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.395757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1154/2317] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:30 2022] PUT /identity/v3/projects/f333b21cf0a44c88a8ee5703c4946fbf/users/e51099834f564c8aa4df9bd11e848a4a/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:30.406838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.407176 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.407480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.435129 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] MFA Rules not processed for user `e51099834f564c8aa4df9bd11e848a4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:30.436332 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:30.502872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1164/2318] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:30 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:30.514575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.516273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:30.572403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba8abbc-4c5a-4e20-a10b-01c9918ceaa8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.576037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.576631 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.576906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.577177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.586237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.587647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.597168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1155/2319] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:30 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:30.676480 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:30.686784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe00f4f-9aaa-4839-bb53-1eccc64b363e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:30.690347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b936a6db-b732-4796-8ed7-add87a8ab088 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:30.690959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936a6db-b732-4796-8ed7-add87a8ab088 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:30.691232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936a6db-b732-4796-8ed7-add87a8ab088 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:30.691500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b936a6db-b732-4796-8ed7-add87a8ab088 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:30.746816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936a6db-b732-4796-8ed7-add87a8ab088 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:30.748240 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b936a6db-b732-4796-8ed7-add87a8ab088 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:30.757142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1165/2320] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:30 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:34.805252 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:34.866621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e58e01-ff91-4f35-b7b8-6793e15f4d80 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:34.871021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:34.871633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:34.871906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:34.872198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] PATH_INFO: `/v3/users/ae3c586feea24c04949956cc00ce2dfc` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:34.873292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae3c586feea24c04949956cc00ce2dfc)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:34.875547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:34.919849 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:34.921440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae3c586feea24c04949956cc00ce2dfc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:34.922197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae3c586feea24c04949956cc00ce2dfc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:34.923037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae3c586feea24c04949956cc00ce2dfc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:34.932288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae3c586feea24c04949956cc00ce2dfc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:34.938982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae3c586feea24c04949956cc00ce2dfc'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:34.956462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1156/2321] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:34 2022] DELETE /identity/v3/users/ae3c586feea24c04949956cc00ce2dfc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:34.960956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:34.961474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:34.961593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:34.969382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:34.980574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:34.981795 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:35.047848 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1166/2322] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:35.058227 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:35.059669 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:35.116722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a888f-8390-46f8-b34d-0cd9f1cde3bb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:35.120366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:35.120965 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:35.121241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:35.121539 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:35.130727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:35.132158 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:35.141504 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1157/2323] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:35.407481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:35.408918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:35.421819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff69bee-db64-48e6-9869-3b934c54f847 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:35.426311 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:35.427059 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:35.427340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:35.427602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] PATH_INFO: `/v3/projects/e4ffab40870d44fc9a63dab203c0b19b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:35.428708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4ffab40870d44fc9a63dab203c0b19b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:35.430706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:35.476272 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4ffab40870d44fc9a63dab203c0b19b', 'request_id': 'req-f9c799dd-2560-4579-a574-697904173b14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.482655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4ffab40870d44fc9a63dab203c0b19b', 'request_id': 'req-f9c799dd-2560-4579-a574-697904173b14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.490501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4ffab40870d44fc9a63dab203c0b19b', 'request_id': 'req-f9c799dd-2560-4579-a574-697904173b14'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.501132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] The token cache is being invalidate because project e4ffab40870d44fc9a63dab203c0b19b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:35.501451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.502125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.503626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1167/2324] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:35 2022] DELETE /identity/v3/projects/e4ffab40870d44fc9a63dab203c0b19b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:35.507646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:35.517118 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2544f3-7349-4f82-84d9-88dee10614ad service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:35.572949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:35.573611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:35.573891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:35.574162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] PATH_INFO: `/v3/users/e51099834f564c8aa4df9bd11e848a4a` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:35.575277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e51099834f564c8aa4df9bd11e848a4a)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:35.577237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:35.623503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e51099834f564c8aa4df9bd11e848a4a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.624201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e51099834f564c8aa4df9bd11e848a4a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.625078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e51099834f564c8aa4df9bd11e848a4a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.631978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e51099834f564c8aa4df9bd11e848a4a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.638956 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ccac196-68c7-4977-8be2-21d470d9bfe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e51099834f564c8aa4df9bd11e848a4a'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:35.643250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:35.658346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1158/2325] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:35 2022] DELETE /identity/v3/users/e51099834f564c8aa4df9bd11e848a4a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:35.934285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:35.935926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:35.956302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c799dd-2560-4579-a574-697904173b14 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.015440 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.016110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.016259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.016526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] PATH_INFO: `/v3/projects/f333b21cf0a44c88a8ee5703c4946fbf` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.017654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f333b21cf0a44c88a8ee5703c4946fbf)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.019553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.053751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:36.067406 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f333b21cf0a44c88a8ee5703c4946fbf', 'request_id': 'req-655f00ef-49f2-40cb-951c-eb63871c1625'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:36.073474 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f333b21cf0a44c88a8ee5703c4946fbf', 'request_id': 'req-655f00ef-49f2-40cb-951c-eb63871c1625'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:36.080409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f333b21cf0a44c88a8ee5703c4946fbf', 'request_id': 'req-655f00ef-49f2-40cb-951c-eb63871c1625'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:36.090583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] The token cache is being invalidate because project f333b21cf0a44c88a8ee5703c4946fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:36.090879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:36.091744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-655f00ef-49f2-40cb-951c-eb63871c1625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:36.093354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1168/2326] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:35 2022] DELETE /identity/v3/projects/f333b21cf0a44c88a8ee5703c4946fbf => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:36.098237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.099910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.099910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.118992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:36.120168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:36.178253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1159/2327] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:36.185942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.186238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.186520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.205917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:36.207087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:36.215719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1169/2328] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:36.219904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.230156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdf07b4-f9ec-47e3-bd15-29ba97f4051c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.234690 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.235278 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.235555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.235827 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.236468 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:36.236769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.238402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.249989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1160/2329] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:36.254235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.263593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3092b2-af40-4607-97f7-39643a47cf26 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.269116 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.269708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.269993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.270270 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.271544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.273452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.301930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1170/2330] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:36.306233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.316320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb00dcc-532e-4796-8d6b-4fbc6d636dd8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.374804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.375595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.375875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.376150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.376786 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.378741 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.400924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1161/2331] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:36.405163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.416170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690861a7-03a0-4045-9ec0-695e91e53d06 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.420773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.421393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.421674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.421947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.422592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.424193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.433875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1171/2332] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:36.437825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.447909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301716d-b71e-42c5-9c95-d7fb28656250 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.452462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5b9cb-f565-417f-b159-6748cf6286eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.453040 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b9cb-f565-417f-b159-6748cf6286eb admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.453329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b9cb-f565-417f-b159-6748cf6286eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.453620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5b9cb-f565-417f-b159-6748cf6286eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.454253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5b9cb-f565-417f-b159-6748cf6286eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.455838 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5b9cb-f565-417f-b159-6748cf6286eb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.466726 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1162/2333] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:36.470739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.480990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be0a61c-ec80-40db-af3f-15ac852e5b64 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.485558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.486193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.486471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.486742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] PATH_INFO: `/v3/projects/5679a09a1d934c92ad78bbc376728b63/users/3e2582d83109404db0398bb2011f041d/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.501170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5679a09a1d934c92ad78bbc376728b63, user_id=3e2582d83109404db0398bb2011f041d, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.503856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.516670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1172/2334] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:36 2022] PUT /identity/v3/projects/5679a09a1d934c92ad78bbc376728b63/users/3e2582d83109404db0398bb2011f041d/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:36.527425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.527646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.527940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.554775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] MFA Rules not processed for user `3e2582d83109404db0398bb2011f041d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:36.555992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:36.595303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:36.620104 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1163/2335] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:36.624966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.687872 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17242c54-d6f2-4222-93ab-575f9536952b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.692662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.693259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.693563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.693838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.694469 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.696371 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.721938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1173/2336] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/projects => generated 389 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:36.726604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.737015 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3db82c0-9f09-4030-89c4-ada13463a04c None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.792428 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.793003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.793285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.793590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.794211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.796132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.821608 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1164/2337] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:36 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:36.824257 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.834497 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb944f47-a559-4221-86db-76a93401feb5 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.839039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.839603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.839870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.840146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.840773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.842396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.853217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1174/2338] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:36.857396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.867779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0055352c-e56b-428b-a0bb-f31360455bec admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.872368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.873003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.873334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.873621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] PATH_INFO: `/v3/projects/f269c7aa01e945539c005f2a7049b06f/users/2cb6c26efe9245cdb2a22c4842777298/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.888595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f269c7aa01e945539c005f2a7049b06f, user_id=2cb6c26efe9245cdb2a22c4842777298, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.891351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.904934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1165/2339] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:36 2022] PUT /identity/v3/projects/f269c7aa01e945539c005f2a7049b06f/users/2cb6c26efe9245cdb2a22c4842777298/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:36.908842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:36.919343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1762d-31d7-480a-9b1d-ecf4d1ad3512 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:36.944438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:36.976135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7693cd7-7d10-4b4b-a2dd-f8168d3f1cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:36.976743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693cd7-7d10-4b4b-a2dd-f8168d3f1cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:36.977019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693cd7-7d10-4b4b-a2dd-f8168d3f1cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:36.977289 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7693cd7-7d10-4b4b-a2dd-f8168d3f1cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:36.977950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7693cd7-7d10-4b4b-a2dd-f8168d3f1cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:36.979545 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7693cd7-7d10-4b4b-a2dd-f8168d3f1cbe admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:36.989726 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1175/2340] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:36.994009 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:37.004365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537cc01-10cc-4873-b259-b4f712e73d0e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:37.009552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:37.010199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.010475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.011262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] PATH_INFO: `/v3/projects/f269c7aa01e945539c005f2a7049b06f/users/2cb6c26efe9245cdb2a22c4842777298/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.014018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f269c7aa01e945539c005f2a7049b06f, user_id=2cb6c26efe9245cdb2a22c4842777298, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:37.019339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:37.030579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1166/2341] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:36 2022] PUT /identity/v3/projects/f269c7aa01e945539c005f2a7049b06f/users/2cb6c26efe9245cdb2a22c4842777298/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:37.041857 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.042492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.042492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.071030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] MFA Rules not processed for user `2cb6c26efe9245cdb2a22c4842777298`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:37.072232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.140062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1176/2342] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:37 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:37.150899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:37.152692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.208953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd1591-8d24-42fa-b5b3-f09dc001bda7 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:37.212511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d83034-2248-403c-a85b-8f387c7c638a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:37.213093 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d83034-2248-403c-a85b-8f387c7c638a service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.213399 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d83034-2248-403c-a85b-8f387c7c638a service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.213670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d83034-2248-403c-a85b-8f387c7c638a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.222389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d83034-2248-403c-a85b-8f387c7c638a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:37.223903 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d83034-2248-403c-a85b-8f387c7c638a service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:37.233057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1167/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:37.367397 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:37.429372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15cdf6-b763-4822-a319-4b3cddff6dfc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:37.433865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:37.434484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.434763 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.435036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] PATH_INFO: `/v3/users/3e2582d83109404db0398bb2011f041d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.436146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e2582d83109404db0398bb2011f041d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:37.438114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:37.484430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e2582d83109404db0398bb2011f041d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:37.488039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:37.492758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e2582d83109404db0398bb2011f041d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:37.493512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e2582d83109404db0398bb2011f041d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:37.502033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e2582d83109404db0398bb2011f041d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:37.502664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e2582d83109404db0398bb2011f041d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:37.519810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1177/2344] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:37 2022] DELETE /identity/v3/users/3e2582d83109404db0398bb2011f041d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:37.524612 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.524901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.525176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.544478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:37.545715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.575788 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:37.605722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1168/2345] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:37.615911 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:37.617357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.674983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5020c2ea-ee30-4ec9-949c-f31b946e878b admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:37.678858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:37.679450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.679724 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.679991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.688851 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:37.690348 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:37.699894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1178/2346] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:37.967199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:37.968632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:37.981478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322a7504-889f-42dd-831c-a819a71248d2 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:37.986591 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:37.987219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:37.987555 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:37.987826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] PATH_INFO: `/v3/projects/5679a09a1d934c92ad78bbc376728b63` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:37.988942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5679a09a1d934c92ad78bbc376728b63)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:37.990982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.036542 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5679a09a1d934c92ad78bbc376728b63', 'request_id': 'req-5e3d8e0c-9aee-4738-a204-caf27902538c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.043233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5679a09a1d934c92ad78bbc376728b63', 'request_id': 'req-5e3d8e0c-9aee-4738-a204-caf27902538c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.049483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5679a09a1d934c92ad78bbc376728b63', 'request_id': 'req-5e3d8e0c-9aee-4738-a204-caf27902538c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.060222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] The token cache is being invalidate because project 5679a09a1d934c92ad78bbc376728b63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:38.060510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.061140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.062623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1169/2347] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:37 2022] DELETE /identity/v3/projects/5679a09a1d934c92ad78bbc376728b63 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:38.066563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:38.076524 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f748c-640b-45df-8b3b-39d14e81fdd4 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:38.133476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:38.134251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.134592 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.134865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] PATH_INFO: `/v3/users/2cb6c26efe9245cdb2a22c4842777298` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.136002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb6c26efe9245cdb2a22c4842777298)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:38.138098 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.183498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb6c26efe9245cdb2a22c4842777298'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.191019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb6c26efe9245cdb2a22c4842777298'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.191899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb6c26efe9245cdb2a22c4842777298'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.199508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb6c26efe9245cdb2a22c4842777298'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.200130 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-678da1d2-0a56-485e-9075-c61ed5ff1fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb6c26efe9245cdb2a22c4842777298'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.207753 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:38.220445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1179/2348] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:38 2022] DELETE /identity/v3/users/2cb6c26efe9245cdb2a22c4842777298 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:38.485231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:38.486750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:38.507426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3d8e0c-9aee-4738-a204-caf27902538c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:38.563588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:38.564203 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.564478 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.564752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] PATH_INFO: `/v3/projects/f269c7aa01e945539c005f2a7049b06f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.565873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f269c7aa01e945539c005f2a7049b06f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:38.567756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.614650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f269c7aa01e945539c005f2a7049b06f', 'request_id': 'req-5fff04b4-2a7d-4468-b670-c2ddb05fff00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.622126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f269c7aa01e945539c005f2a7049b06f', 'request_id': 'req-5fff04b4-2a7d-4468-b670-c2ddb05fff00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.629260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f269c7aa01e945539c005f2a7049b06f', 'request_id': 'req-5fff04b4-2a7d-4468-b670-c2ddb05fff00'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.630466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:38.644711 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] The token cache is being invalidate because project f269c7aa01e945539c005f2a7049b06f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:38.644996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.645695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-5fff04b4-2a7d-4468-b670-c2ddb05fff00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:38.647275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1170/2349] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:38 2022] DELETE /identity/v3/projects/f269c7aa01e945539c005f2a7049b06f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:38.652374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.652662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.652938 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.673591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:38.675620 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:38.735752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1180/2350] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:38.744161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45444829-c481-4883-818c-740d29a6c43e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.744445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45444829-c481-4883-818c-740d29a6c43e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.744716 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45444829-c481-4883-818c-740d29a6c43e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.764230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-45444829-c481-4883-818c-740d29a6c43e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:38.765502 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-45444829-c481-4883-818c-740d29a6c43e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:38.773121 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1171/2351] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:38.777409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:38.786950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d98931-949c-46c5-b253-5fc9cb1db181 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:38.792145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:38.792740 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.793019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.793291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.793956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:38.794259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:38.795850 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.807859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1181/2352] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:38.812088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45444829-c481-4883-818c-740d29a6c43e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:38.821818 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45444829-c481-4883-818c-740d29a6c43e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:38.826443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:38.827030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.827303 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.827571 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.828185 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:38.830123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.857052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1172/2353] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:38 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:38.861351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:38.871895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339158-9d20-4165-8622-f34d57cdae1d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:38.928034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:38.929142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.929630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.929905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.930533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:38.932439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.953235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1182/2354] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:38 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:38.957378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:38.967429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fef4d4-317b-40c0-adf2-4522a9316374 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:38.971971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:38.972554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:38.972829 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:38.973094 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:38.973765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:38.975363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:38.985477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1173/2355] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:38.989173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.000089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd253013-4097-4d4e-8cc8-6e47e449127c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.004891 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98528ab-f9ac-4466-b02d-13476f4f615f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.005515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98528ab-f9ac-4466-b02d-13476f4f615f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.005797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98528ab-f9ac-4466-b02d-13476f4f615f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.006072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98528ab-f9ac-4466-b02d-13476f4f615f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.006713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98528ab-f9ac-4466-b02d-13476f4f615f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.008302 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98528ab-f9ac-4466-b02d-13476f4f615f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.018793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1183/2356] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:39.023086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.033120 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98908383-64c3-4a60-b942-347bbd4d88bf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.037930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.039184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.039464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.039736 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] PATH_INFO: `/v3/projects/2b5a5407daea40f8a985f0dc8d4b30fb/users/349488bb53ba4de7959479deb74a8b87/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.054635 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5a5407daea40f8a985f0dc8d4b30fb, user_id=349488bb53ba4de7959479deb74a8b87, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.057333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.070272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1174/2357] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:39 2022] PUT /identity/v3/projects/2b5a5407daea40f8a985f0dc8d4b30fb/users/349488bb53ba4de7959479deb74a8b87/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:39.081384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.081868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.082006 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.111381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] MFA Rules not processed for user `349488bb53ba4de7959479deb74a8b87`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:39.112634 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:39.157853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:39.179510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1184/2358] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:39.183852 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.246170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344f457-cd37-4abf-87fe-509455a8e6ef admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.251018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.251620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.251894 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.252161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.252778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.254770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.282241 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1175/2359] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:39 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:39.286174 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.297615 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39600-f5f1-47f9-a582-429d359d318a None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.352625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.353227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.353541 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.353818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.354455 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.356378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.377581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1185/2360] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:39 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:39.381535 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.391588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e87ed5-5085-47f8-86a4-c0c3ff19118c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.396411 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.397034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.397338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.398107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.398803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.400754 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.411575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1176/2361] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:39.416441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.426147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210cbbb-65c4-483f-8966-07a65aa19df1 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.430758 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.431334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.431607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.431875 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] PATH_INFO: `/v3/projects/b9e3a2a5e9e6495bbe2be939646b23df/users/ad1876c6e74240c493217b339f809eea/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.446607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e3a2a5e9e6495bbe2be939646b23df, user_id=ad1876c6e74240c493217b339f809eea, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.449248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.462717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1186/2362] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:39 2022] PUT /identity/v3/projects/b9e3a2a5e9e6495bbe2be939646b23df/users/ad1876c6e74240c493217b339f809eea/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:39.466677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.476740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593fccd-a416-4469-a4a7-6a731b02136e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.531794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43adbe7-c9ac-4387-ae2a-03c504a4b2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.532466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43adbe7-c9ac-4387-ae2a-03c504a4b2ce admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.532657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43adbe7-c9ac-4387-ae2a-03c504a4b2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.532928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43adbe7-c9ac-4387-ae2a-03c504a4b2ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.533593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43adbe7-c9ac-4387-ae2a-03c504a4b2ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.535181 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43adbe7-c9ac-4387-ae2a-03c504a4b2ce admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.536383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:39.546862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1177/2363] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:39.550775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.561273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c99c24-b925-4c12-b726-f20771214ae3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.565884 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.566526 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.566825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.567093 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] PATH_INFO: `/v3/projects/b9e3a2a5e9e6495bbe2be939646b23df/users/ad1876c6e74240c493217b339f809eea/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.568926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e3a2a5e9e6495bbe2be939646b23df, user_id=ad1876c6e74240c493217b339f809eea, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.572351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.582337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1187/2364] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:39 2022] PUT /identity/v3/projects/b9e3a2a5e9e6495bbe2be939646b23df/users/ad1876c6e74240c493217b339f809eea/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:39.592962 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.593251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.593562 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.621320 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] MFA Rules not processed for user `ad1876c6e74240c493217b339f809eea`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:39.622552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:39.690353 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1178/2365] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:39.701908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:39.703611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:39.759609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1caa9f-cb45-49ed-bd69-dc0f7dd943f9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:39.763338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059cd36-d18b-4cc9-9216-690a4b340a9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:39.763961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059cd36-d18b-4cc9-9216-690a4b340a9d service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:39.764242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059cd36-d18b-4cc9-9216-690a4b340a9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:39.764517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059cd36-d18b-4cc9-9216-690a4b340a9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:39.774086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059cd36-d18b-4cc9-9216-690a4b340a9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:39.775510 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059cd36-d18b-4cc9-9216-690a4b340a9d service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:39.784730 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1188/2366] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:40.154402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:40.216250 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6bac5-edb9-4866-953d-841002cf56f7 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:40.220717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:40.221352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:40.221629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:40.221893 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] PATH_INFO: `/v3/users/349488bb53ba4de7959479deb74a8b87` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:40.222997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=349488bb53ba4de7959479deb74a8b87)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:40.224885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:40.270955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349488bb53ba4de7959479deb74a8b87'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.271621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349488bb53ba4de7959479deb74a8b87'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.272231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349488bb53ba4de7959479deb74a8b87'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.279598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349488bb53ba4de7959479deb74a8b87'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.286714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349488bb53ba4de7959479deb74a8b87'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.299959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:40.304721 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1179/2367] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:40 2022] DELETE /identity/v3/users/349488bb53ba4de7959479deb74a8b87 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:40.309847 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:40.310367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:40.310507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:40.329666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:40.330859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:40.368194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:40.390170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1189/2368] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:43:40.400454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:40.401926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:40.458339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b40eeb-5250-48ac-b7aa-dffcc7ba62bf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:40.462620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:40.463222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:40.463511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:40.463772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:40.472673 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:40.474125 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:40.483454 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1180/2369] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:40.758471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:40.759980 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:40.778509 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef97977-6b7f-4338-8240-118a0b1d4996 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:40.783131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:40.783770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:40.784029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:40.784299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] PATH_INFO: `/v3/projects/2b5a5407daea40f8a985f0dc8d4b30fb` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:40.785457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b5a5407daea40f8a985f0dc8d4b30fb)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:40.787352 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:40.833933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b5a5407daea40f8a985f0dc8d4b30fb', 'request_id': 'req-a7c9326d-4253-48ad-9856-25235c1ed7b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.840110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b5a5407daea40f8a985f0dc8d4b30fb', 'request_id': 'req-a7c9326d-4253-48ad-9856-25235c1ed7b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.847416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b5a5407daea40f8a985f0dc8d4b30fb', 'request_id': 'req-a7c9326d-4253-48ad-9856-25235c1ed7b0'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.857554 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] The token cache is being invalidate because project 2b5a5407daea40f8a985f0dc8d4b30fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:40.857860 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.858475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.859934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1190/2370] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:40 2022] DELETE /identity/v3/projects/2b5a5407daea40f8a985f0dc8d4b30fb => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:40.863955 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:40.874492 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350716f3-a1e6-4847-8fd5-17642db86e99 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:40.932901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:40.933659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:40.933940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:40.934212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] PATH_INFO: `/v3/users/ad1876c6e74240c493217b339f809eea` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:40.935321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad1876c6e74240c493217b339f809eea)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:40.937226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:40.982291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad1876c6e74240c493217b339f809eea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.982936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad1876c6e74240c493217b339f809eea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.983544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad1876c6e74240c493217b339f809eea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.990339 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad1876c6e74240c493217b339f809eea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:40.998874 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-978d22fa-0256-4bf2-9321-919c96dc0296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad1876c6e74240c493217b339f809eea'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:41.016022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1181/2371] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:40 2022] DELETE /identity/v3/users/ad1876c6e74240c493217b339f809eea => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:41.291045 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.292493 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:41.307078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9326d-4253-48ad-9856-25235c1ed7b0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.363734 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.364375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.364625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.364898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] PATH_INFO: `/v3/projects/b9e3a2a5e9e6495bbe2be939646b23df` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.366043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e3a2a5e9e6495bbe2be939646b23df)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.368016 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.415735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e3a2a5e9e6495bbe2be939646b23df', 'request_id': 'req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:41.421959 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e3a2a5e9e6495bbe2be939646b23df', 'request_id': 'req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:41.429264 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e3a2a5e9e6495bbe2be939646b23df', 'request_id': 'req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:41.433944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:41.442295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] The token cache is being invalidate because project b9e3a2a5e9e6495bbe2be939646b23df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:41.442808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:41.443562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-a3f5bc0f-7169-4a4c-bb4d-92fa94433fbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:41.445094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1191/2372] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:41 2022] DELETE /identity/v3/projects/b9e3a2a5e9e6495bbe2be939646b23df => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:41.449939 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.450254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.450531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.470453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:41.471657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:41.488692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:41.533831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1182/2373] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:41.541522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.541834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.542105 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.561908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:41.563106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:41.571550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1192/2374] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:41.575730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.585335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f7d17f-d541-402b-ad03-2ee87772e980 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.590145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.590731 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.591007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.591277 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.591913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:41.592239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.593869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.605344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1183/2375] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:43:41.609517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.619081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d420b32b-e9bd-48f8-a08a-038aeef49a68 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.623640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.624484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.624757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.625051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.625708 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.627589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.654313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1193/2376] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:41 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:43:41.658405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.668456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e21c9-b9b3-4583-b64f-da8aea00031f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.724426 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.725014 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.725288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.726131 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.726753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.728670 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.750506 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1184/2377] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:41 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:41.754539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.764663 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeb5c33-a82f-4aae-9d71-fbb52ad40f77 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.769183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.769805 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.770097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.770363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.771014 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.772588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.782219 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1194/2378] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:41.786151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.796724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e74304f-21bf-4e42-b182-1528fcbd3ed3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.801276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d167f-477c-4666-9075-10c4d040db5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.801887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d167f-477c-4666-9075-10c4d040db5d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.802168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d167f-477c-4666-9075-10c4d040db5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.802443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d167f-477c-4666-9075-10c4d040db5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.803072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d167f-477c-4666-9075-10c4d040db5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.805354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d167f-477c-4666-9075-10c4d040db5d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.815912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1185/2379] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:41.820128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:41.830054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c8a846-8fe3-4dce-8010-32898d5eddd2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:41.834742 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:41.835494 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.835775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.836050 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] PATH_INFO: `/v3/projects/4ad0c9491c2a414785c79c5232ee79a8/users/303ebf600a004a80a09debf0bfda8de7/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.850487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad0c9491c2a414785c79c5232ee79a8, user_id=303ebf600a004a80a09debf0bfda8de7, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:41.853676 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:41.866985 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1195/2380] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:41 2022] PUT /identity/v3/projects/4ad0c9491c2a414785c79c5232ee79a8/users/303ebf600a004a80a09debf0bfda8de7/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:41.932054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:41.932629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:41.932629 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:41.960697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] MFA Rules not processed for user `303ebf600a004a80a09debf0bfda8de7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:41.961940 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:42.024477 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1186/2381] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:42.036443 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:42.038193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:42.093831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca8b74-9567-4d52-b698-30df77c5327e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:42.097448 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04334480-db00-4376-a3b3-0d1a8d922ff4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:42.098039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04334480-db00-4376-a3b3-0d1a8d922ff4 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:42.098312 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04334480-db00-4376-a3b3-0d1a8d922ff4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:42.098578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04334480-db00-4376-a3b3-0d1a8d922ff4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:42.106881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04334480-db00-4376-a3b3-0d1a8d922ff4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:42.108319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04334480-db00-4376-a3b3-0d1a8d922ff4 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:42.117709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1196/2382] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:42.140217 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:42.159366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:42.204413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eca171-e812-44e0-afd4-fd9efd6594cb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:42.208950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:42.209606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:42.209875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:42.210254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] PATH_INFO: `/v3/users/303ebf600a004a80a09debf0bfda8de7` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:42.211308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303ebf600a004a80a09debf0bfda8de7)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:42.213207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:42.271208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303ebf600a004a80a09debf0bfda8de7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:42.271902 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303ebf600a004a80a09debf0bfda8de7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:42.272570 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303ebf600a004a80a09debf0bfda8de7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:42.279959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303ebf600a004a80a09debf0bfda8de7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:42.287920 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303ebf600a004a80a09debf0bfda8de7'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:42.305673 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1187/2383] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:42 2022] DELETE /identity/v3/users/303ebf600a004a80a09debf0bfda8de7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:42.310097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:42.310438 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:42.310745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:42.332582 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:42.333833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:42.392099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1197/2384] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:42.402344 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:42.403742 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:42.460771 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4be6dd-bdee-4b74-8b4d-50741f215c45 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:42.464640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f261c12-d632-4e5e-932a-70bbbaabf21a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:42.465236 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f261c12-d632-4e5e-932a-70bbbaabf21a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:42.465546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f261c12-d632-4e5e-932a-70bbbaabf21a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:42.465848 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f261c12-d632-4e5e-932a-70bbbaabf21a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:42.474698 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f261c12-d632-4e5e-932a-70bbbaabf21a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:42.476834 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f261c12-d632-4e5e-932a-70bbbaabf21a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:42.486095 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1188/2385] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:43:43.004790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.006326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:43.020386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f05474f-9941-4ed8-b80b-e3a66ea8ebb3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.026146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.026953 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.027268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.027492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] PATH_INFO: `/v3/projects/4ad0c9491c2a414785c79c5232ee79a8` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.028778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad0c9491c2a414785c79c5232ee79a8)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.032427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.069901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:43.083821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad0c9491c2a414785c79c5232ee79a8', 'request_id': 'req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:43.090814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad0c9491c2a414785c79c5232ee79a8', 'request_id': 'req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:43.099055 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad0c9491c2a414785c79c5232ee79a8', 'request_id': 'req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:43.110706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] The token cache is being invalidate because project 4ad0c9491c2a414785c79c5232ee79a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:43.110997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:43.111621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c84a362d-1b2a-4aaf-96c9-3f27cdbe3f09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:43.113141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1198/2386] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:43 2022] DELETE /identity/v3/projects/4ad0c9491c2a414785c79c5232ee79a8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:43.118128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.118596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.118695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.138543 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:43.139737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:43.174346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:43.200691 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1189/2387] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:43.208262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.208555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.208841 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.229126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:43.230369 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:43.238250 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1199/2388] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:43.242453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.252271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6101e886-96e2-4d0b-8653-a6a9a99fc654 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.257334 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.258151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.258432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.258701 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.259336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:43.259643 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.261243 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.274730 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1190/2389] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:43.278954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.288266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5e127-c36e-48a4-bed7-7ea8ea2ac34b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.292982 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.293617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.293894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.294166 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.294920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.296811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.324532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1200/2390] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:43.328728 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.339247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ffe083-bb1f-4d3b-a1c7-c3feee694a55 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.394974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.395655 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.395938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.396208 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.396825 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.399348 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.420200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1191/2391] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:43.424447 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.434770 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc044e9-cc8c-40b7-a7da-fcdac570bb66 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.439242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.439827 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.440099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.440365 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.440993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.442605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.452705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1201/2392] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:43.456613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.467119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d4a60-6cb6-4761-866d-c7c946a498be admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.471648 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9f1de-9071-417c-b123-90745fc60e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.472228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9f1de-9071-417c-b123-90745fc60e78 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.472505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9f1de-9071-417c-b123-90745fc60e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.472776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9f1de-9071-417c-b123-90745fc60e78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.473436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9f1de-9071-417c-b123-90745fc60e78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.475006 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9f1de-9071-417c-b123-90745fc60e78 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.485439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1192/2393] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:43.489301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.499375 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18bcde-87d8-40b2-a543-66f13e01a82d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.504036 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.504603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.504879 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.505147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] PATH_INFO: `/v3/projects/57c1269f1965410084804b34207c2a02/users/118a7efe68454fff8e0d25a337366763/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.521741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c1269f1965410084804b34207c2a02, user_id=118a7efe68454fff8e0d25a337366763, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.524985 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.537278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1202/2394] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:43 2022] PUT /identity/v3/projects/57c1269f1965410084804b34207c2a02/users/118a7efe68454fff8e0d25a337366763/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:43.547563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.547914 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.548190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.575924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] MFA Rules not processed for user `118a7efe68454fff8e0d25a337366763`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:43.577161 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:43.640434 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1193/2395] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:43.645090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.705709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822f8eba-578c-4f91-bb81-b7ba6711968c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.710904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.711506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.711786 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.712056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.712681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.714635 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.741151 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1203/2396] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:43.745333 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.756053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7173b6-a3ed-4762-a53e-85c420323928 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.787912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:43.813769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.814368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.814646 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.814913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.815529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.817470 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.839150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1194/2397] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:43.843354 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.854574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b50ea18-0168-4620-a4df-3ce1443d41bd admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.859368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.859966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.860246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.860519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.861158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.862821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.874902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1204/2398] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:43.879137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.889135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027608cd-dabb-451e-8d20-74d5ec238b52 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.893756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b038f-90b1-450e-b0f3-b4f767c3c31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.894342 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b038f-90b1-450e-b0f3-b4f767c3c31c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.894622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b038f-90b1-450e-b0f3-b4f767c3c31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.894891 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b038f-90b1-450e-b0f3-b4f767c3c31c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.895527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b038f-90b1-450e-b0f3-b4f767c3c31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.897114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b038f-90b1-450e-b0f3-b4f767c3c31c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.907440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1195/2399] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:43.911567 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:43.922120 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fae0ec7-c994-408c-addd-cce799a2ca71 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:43.926877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:43.927516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.927791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.928060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] PATH_INFO: `/v3/projects/d28836b3ffba482dadafb4082b12cbc3/users/8e3f955e54f54f2c98eceb5888987711/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:43.942970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d28836b3ffba482dadafb4082b12cbc3, user_id=8e3f955e54f54f2c98eceb5888987711, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:43.946315 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:43.959610 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1205/2400] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:43 2022] PUT /identity/v3/projects/d28836b3ffba482dadafb4082b12cbc3/users/8e3f955e54f54f2c98eceb5888987711/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:43:43.970904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:43.971192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:43.971467 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.000632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] MFA Rules not processed for user `8e3f955e54f54f2c98eceb5888987711`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:44.001968 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:44.063949 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1196/2401] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:44.068445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.074525 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:44.130772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d372494-5b58-4aa7-b8a1-eaf7c5f3e6d3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.136651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.137249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.137557 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.137831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.138467 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.140366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.166669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1206/2402] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:44 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:44.170807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.181483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dccb1-6b05-41bf-a59e-2825d7faaadb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.236778 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.237403 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.237678 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.237941 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.238548 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.240458 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.264361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1197/2403] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:44 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:44.268441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.279589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57d907d-c474-4589-8284-a89945eafee9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.284208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.284802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.285081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.285498 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.286137 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.287731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.298540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1207/2404] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:44.302708 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.312757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe8608-4005-4b65-9b56-9442e1284cdf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.317382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.317954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.318231 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.318499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] PATH_INFO: `/v3/projects/71e4fb58a2a746888f73e541e36bc305/users/42a0d51261bd4e928cbf4c22ad96d204/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.334218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e4fb58a2a746888f73e541e36bc305, user_id=42a0d51261bd4e928cbf4c22ad96d204, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.337882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.350027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1198/2405] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:44 2022] PUT /identity/v3/projects/71e4fb58a2a746888f73e541e36bc305/users/42a0d51261bd4e928cbf4c22ad96d204/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:44.354282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.364212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374a74f2-c41a-427c-836d-6bd79ca3d571 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.420648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167dc8dd-0dff-480e-a49d-fc3d52733800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.421239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dc8dd-0dff-480e-a49d-fc3d52733800 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.421575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dc8dd-0dff-480e-a49d-fc3d52733800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.421834 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167dc8dd-0dff-480e-a49d-fc3d52733800 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.422462 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dc8dd-0dff-480e-a49d-fc3d52733800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.424076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167dc8dd-0dff-480e-a49d-fc3d52733800 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.434317 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1208/2406] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:44.438391 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.449657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f59a0-70e7-46d6-9e36-55b197bcc020 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.454232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.454866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.455140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.455408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] PATH_INFO: `/v3/projects/71e4fb58a2a746888f73e541e36bc305/users/42a0d51261bd4e928cbf4c22ad96d204/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.457283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71e4fb58a2a746888f73e541e36bc305, user_id=42a0d51261bd4e928cbf4c22ad96d204, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.459982 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.463038 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:44.471003 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1199/2407] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:44 2022] PUT /identity/v3/projects/71e4fb58a2a746888f73e541e36bc305/users/42a0d51261bd4e928cbf4c22ad96d204/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:44.481033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.481368 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.481656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.511199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] MFA Rules not processed for user `42a0d51261bd4e928cbf4c22ad96d204`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:44.512417 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:44.575597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:44.579695 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1209/2408] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:44.591016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:44.592714 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:44.649924 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25b4b65-dc38-4a37-9818-9b6d3aafba22 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:44.653596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:44.654188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:44.654466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:44.654734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:44.710791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:44.712211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:44.721686 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1200/2409] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:46.248645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:46.250357 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:46.266761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685f54-edfb-4594-b0aa-833c1b429ee3 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:46.270674 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:46.271279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:46.271555 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:46.271826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:46.334558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:46.336731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:46.348091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1210/2410] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:46.927706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:46.937001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c512f883-bf0c-4f7a-b284-5703660838c2 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:46.940642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca8eba2-f987-418a-b891-8e1de7dea659 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:46.941235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8eba2-f987-418a-b891-8e1de7dea659 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:46.941544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8eba2-f987-418a-b891-8e1de7dea659 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:46.941815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca8eba2-f987-418a-b891-8e1de7dea659 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:46.951500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8eba2-f987-418a-b891-8e1de7dea659 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:46.952918 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca8eba2-f987-418a-b891-8e1de7dea659 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:46.962899 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1201/2411] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:50.827759 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:50.888261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5640d8-5c88-441e-8538-66f3fd0481a6 service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:50.892898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:50.893573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:50.893853 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:50.894144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] PATH_INFO: `/v3/users/118a7efe68454fff8e0d25a337366763` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:50.895288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=118a7efe68454fff8e0d25a337366763)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:50.897442 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:50.942627 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '118a7efe68454fff8e0d25a337366763'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:50.950522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '118a7efe68454fff8e0d25a337366763'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:50.951131 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '118a7efe68454fff8e0d25a337366763'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:50.957873 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '118a7efe68454fff8e0d25a337366763'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:50.958488 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '118a7efe68454fff8e0d25a337366763'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:50.976134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1211/2412] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:50 2022] DELETE /identity/v3/users/118a7efe68454fff8e0d25a337366763 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:50.980604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:50.981089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:50.981189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:51.000049 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:51.001233 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:51.060053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1202/2413] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:51.069628 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:51.071083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:51.091605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:51.130651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39672717-d5f3-4f8e-8c96-c62c472c3682 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:51.134994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:51.135598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:51.135880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:51.136156 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:51.145118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:51.146577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:51.156386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1212/2414] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:43:51.419652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:51.421089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:51.434758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84419586-9fee-4102-9540-2c47acb91f41 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:51.439541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:51.440215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:51.440493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:51.440887 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] PATH_INFO: `/v3/projects/57c1269f1965410084804b34207c2a02` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:51.442016 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c1269f1965410084804b34207c2a02)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:51.444959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:51.481099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:51.493926 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c1269f1965410084804b34207c2a02', 'request_id': 'req-ad785694-19a7-4666-9729-485013214a03'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.501372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c1269f1965410084804b34207c2a02', 'request_id': 'req-ad785694-19a7-4666-9729-485013214a03'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.508447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c1269f1965410084804b34207c2a02', 'request_id': 'req-ad785694-19a7-4666-9729-485013214a03'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.518378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] The token cache is being invalidate because project 57c1269f1965410084804b34207c2a02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:51.518661 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.519279 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.520729 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1203/2415] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:51 2022] DELETE /identity/v3/projects/57c1269f1965410084804b34207c2a02 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:51.525377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:51.534999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b2e1b-743f-4931-8cde-fe0ffc715891 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:51.590926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:51.591558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:51.591842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:51.592118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] PATH_INFO: `/v3/users/8e3f955e54f54f2c98eceb5888987711` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:51.593297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3f955e54f54f2c98eceb5888987711)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:51.595191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:51.642094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3f955e54f54f2c98eceb5888987711'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.649637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3f955e54f54f2c98eceb5888987711'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.650341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3f955e54f54f2c98eceb5888987711'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.657145 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3f955e54f54f2c98eceb5888987711'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.657822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3f955e54f54f2c98eceb5888987711'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:51.674539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1213/2416] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:51 2022] DELETE /identity/v3/users/8e3f955e54f54f2c98eceb5888987711 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:51.946503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:51.947973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:51.961272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad785694-19a7-4666-9729-485013214a03 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.018882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.019563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.019764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.020034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] PATH_INFO: `/v3/projects/d28836b3ffba482dadafb4082b12cbc3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.021128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d28836b3ffba482dadafb4082b12cbc3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.023183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.070659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd28836b3ffba482dadafb4082b12cbc3', 'request_id': 'req-30be4a4d-5520-4054-bba4-a35ca2fc75aa'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.077604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd28836b3ffba482dadafb4082b12cbc3', 'request_id': 'req-30be4a4d-5520-4054-bba4-a35ca2fc75aa'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.083883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd28836b3ffba482dadafb4082b12cbc3', 'request_id': 'req-30be4a4d-5520-4054-bba4-a35ca2fc75aa'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.094306 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] The token cache is being invalidate because project d28836b3ffba482dadafb4082b12cbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:52.094596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.095226 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.096706 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1204/2417] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:51 2022] DELETE /identity/v3/projects/d28836b3ffba482dadafb4082b12cbc3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:52.100618 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.102115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.114908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85d3762-5cee-420f-a38e-4ef225942915 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.153890 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:52.173918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.174428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.174711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.174987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] PATH_INFO: `/v3/users/42a0d51261bd4e928cbf4c22ad96d204` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.176114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a0d51261bd4e928cbf4c22ad96d204)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.178583 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.224511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a0d51261bd4e928cbf4c22ad96d204'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.232147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a0d51261bd4e928cbf4c22ad96d204'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.232811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a0d51261bd4e928cbf4c22ad96d204'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.239988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a0d51261bd4e928cbf4c22ad96d204'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.241275 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-6e74077b-9a69-4263-ac6f-fd659e2ca7ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a0d51261bd4e928cbf4c22ad96d204'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.261296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1214/2418] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:43:52 2022] DELETE /identity/v3/users/42a0d51261bd4e928cbf4c22ad96d204 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:52.526573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.528027 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.541652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be4a4d-5520-4054-bba4-a35ca2fc75aa admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.589777 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:52.601791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.602418 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.602697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.602971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] PATH_INFO: `/v3/projects/71e4fb58a2a746888f73e541e36bc305` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.604084 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71e4fb58a2a746888f73e541e36bc305)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.606068 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.651262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71e4fb58a2a746888f73e541e36bc305', 'request_id': 'req-4ba1acb5-523c-4d4b-8c57-38f51400c114'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.658371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71e4fb58a2a746888f73e541e36bc305', 'request_id': 'req-4ba1acb5-523c-4d4b-8c57-38f51400c114'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.665140 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71e4fb58a2a746888f73e541e36bc305', 'request_id': 'req-4ba1acb5-523c-4d4b-8c57-38f51400c114'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.675374 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] The token cache is being invalidate because project 71e4fb58a2a746888f73e541e36bc305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:43:52.675664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.676302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-4ba1acb5-523c-4d4b-8c57-38f51400c114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:43:52.677837 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1205/2419] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:43:52 2022] DELETE /identity/v3/projects/71e4fb58a2a746888f73e541e36bc305 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:52.683000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.683284 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.683584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.702917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:52.704126 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.763507 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1215/2420] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:52.772928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.773216 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.773526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.794402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:52.795603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:52.803928 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1206/2421] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:52.808072 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.817666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfdf464-7a4b-4851-90fd-87286a4b753d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.822998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.823588 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.823866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.824133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.824772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:43:52.825073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.826696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.838563 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1216/2422] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:43:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:43:52.842867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.852076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b3b9d-ffd8-4644-ba45-2c4cdb2dac13 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.856724 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.857326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.857605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.857875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.858501 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.860378 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.888382 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1207/2423] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:43:52 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:52.892668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:52.902799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727c39f2-0988-4aff-8037-b0a8ca881d15 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:52.946826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:52.961221 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:52.961846 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:52.962128 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:52.962396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:52.963023 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:52.964915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:52.986071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1217/2424] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:43:52 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:43:52.990280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:53.001450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc106ec-e207-4c98-832c-d92b34c2d57c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:53.006053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:53.006873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:53.007154 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:53.007430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:53.008071 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:53.009734 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:53.019801 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1208/2425] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:53.023746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:53.033573 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553cf6c-2815-48ad-9a07-3da8802c570d admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:53.038896 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9824a73f-9383-4429-9dd8-c039a16631f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:53.039483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824a73f-9383-4429-9dd8-c039a16631f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:53.039780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824a73f-9383-4429-9dd8-c039a16631f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:53.040042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9824a73f-9383-4429-9dd8-c039a16631f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:53.040678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824a73f-9383-4429-9dd8-c039a16631f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:53.042288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9824a73f-9383-4429-9dd8-c039a16631f7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:53.052630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1218/2426] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:43:53.057136 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:53.066942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208d92c-5e8c-47f9-bc20-ce81619c2f9e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:53.071715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:53.072354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:53.072636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:53.072910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] PATH_INFO: `/v3/projects/89ee0cdccb8b4f85addbedd782a8bc2b/users/6390fc5166d44c84a149de1700f8565f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:53.087281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ee0cdccb8b4f85addbedd782a8bc2b, user_id=6390fc5166d44c84a149de1700f8565f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:53.089978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:53.103593 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1209/2427] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:43:53 2022] PUT /identity/v3/projects/89ee0cdccb8b4f85addbedd782a8bc2b/users/6390fc5166d44c84a149de1700f8565f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:43:53.114398 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:53.114905 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:53.115028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:53.142796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] MFA Rules not processed for user `6390fc5166d44c84a149de1700f8565f`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:43:53.143993 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:53.205814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1219/2428] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:43:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:43:53.222981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:43:53.224697 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:43:53.303632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521aa9bd-ab3a-4e14-b651-24d00effdc19 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:43:53.307318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c06ba-6b3a-4cd2-8f6f-6f11853de98b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:43:53.307919 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c06ba-6b3a-4cd2-8f6f-6f11853de98b service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:43:53.308202 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c06ba-6b3a-4cd2-8f6f-6f11853de98b service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:43:53.308469 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c06ba-6b3a-4cd2-8f6f-6f11853de98b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:43:53.316564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c06ba-6b3a-4cd2-8f6f-6f11853de98b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:43:53.318020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c06ba-6b3a-4cd2-8f6f-6f11853de98b service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:43:53.322503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:43:53.327532 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1210/2429] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:43:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:03.202789 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:26.816239 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:26.817669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:26.888384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c745c5e-dbba-4732-a00b-34c1ff4055aa None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:26.893172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:26.893799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:26.894077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:26.894345 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:26.896051 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:26.908271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:26.910158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:26.921267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1220/2430] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:36.410144 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:36.410735 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:36.410951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:36.433508 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] MFA Rules not processed for user `d0010af061ae4490a6855639a6825cac`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:36.434751 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:36.492271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1211/2431] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:36.916811 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=32 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:37.021710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:37.023703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:37.081542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d81ea2-af7d-458a-843f-7abc31568139 service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:37.085197 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:37.085826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:37.086099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:37.086366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:37.088887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:37.099043 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:37.100500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:37.109161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1221/2432] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:37.640811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:37.643978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:37.721350 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86b1100-bf5b-497b-a497-4b7fdd4a46a5 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:37.727285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:37.727995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:37.728272 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:37.728565 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:37.730781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:37.743091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:37.745091 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:37.756628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1212/2433] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:38.648015 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:38.650144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:38.668393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11232cc-18b3-4886-8d1e-6b2cb5e978f9 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:38.673465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:38.674067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:38.674340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:38.674782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:38.676886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:38.690760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:38.692347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d75cab4-653d-44b5-bcad-0d531ebd7746 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:38.710114 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1222/2434] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:41.439596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:41.514689 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb00f0e-c6b9-4a5b-bad7-7b04560f2e75 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:41.519694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:41.520650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:41.521122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:41.523111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] PATH_INFO: `/v3/users/6390fc5166d44c84a149de1700f8565f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:41.524676 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6390fc5166d44c84a149de1700f8565f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:41.527107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:41.576322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:41.600355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6390fc5166d44c84a149de1700f8565f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:41.601123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6390fc5166d44c84a149de1700f8565f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:41.602265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6390fc5166d44c84a149de1700f8565f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:41.614781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6390fc5166d44c84a149de1700f8565f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:41.626971 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6390fc5166d44c84a149de1700f8565f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:41.653419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1213/2435] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:44:41 2022] DELETE /identity/v3/users/6390fc5166d44c84a149de1700f8565f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:44:41.657709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:41.658234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:41.658234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:41.682080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:41.683408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:41.755202 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1223/2436] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:41.772282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:41.773806 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:41.849537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178467c2-d704-4d06-b6de-dc009924dc49 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:41.854336 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d57117-7a2c-4e96-9246-2766b3b86f28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:41.854994 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d57117-7a2c-4e96-9246-2766b3b86f28 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:41.855280 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d57117-7a2c-4e96-9246-2766b3b86f28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:41.855563 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d57117-7a2c-4e96-9246-2766b3b86f28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:41.882417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d57117-7a2c-4e96-9246-2766b3b86f28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:41.884001 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d57117-7a2c-4e96-9246-2766b3b86f28 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:41.897494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1214/2437] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:42.380431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.382066 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:42.395918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac7e31-ca6a-4937-9ec3-d006153972a8 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.401208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.402216 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.402589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.403067 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] PATH_INFO: `/v3/projects/89ee0cdccb8b4f85addbedd782a8bc2b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.404495 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ee0cdccb8b4f85addbedd782a8bc2b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.406609 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.471017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ee0cdccb8b4f85addbedd782a8bc2b', 'request_id': 'req-f4b0b67d-36ed-472c-90fa-2e856753aa27'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:42.478520 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ee0cdccb8b4f85addbedd782a8bc2b', 'request_id': 'req-f4b0b67d-36ed-472c-90fa-2e856753aa27'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:42.487206 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ee0cdccb8b4f85addbedd782a8bc2b', 'request_id': 'req-f4b0b67d-36ed-472c-90fa-2e856753aa27'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:42.499341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] The token cache is being invalidate because project 89ee0cdccb8b4f85addbedd782a8bc2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:44:42.499626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:42.500278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-f4b0b67d-36ed-472c-90fa-2e856753aa27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:42.501970 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1224/2438] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:44:42 2022] DELETE /identity/v3/projects/89ee0cdccb8b4f85addbedd782a8bc2b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:42.507210 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.507808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.507808 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.528239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:42.529465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:42.588896 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1215/2439] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:42.597604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.597889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.598165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.617706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:42.619796 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:42.627562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1225/2440] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:42.631767 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.641888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eb418-52d5-4809-9020-4ed1c9b720ac None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.646451 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.647039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.647318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.647588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.648225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:44:42.648529 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.650165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.661430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1216/2441] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:44:42.665636 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.674700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f529ded-9f24-4f99-9b98-204563fcbe55 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.679247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.679831 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.680110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.680381 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.681005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.683642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.688052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:42.712395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1226/2442] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:44:42 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:42.716357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.726112 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63b0a0d-7536-4060-a655-c91333c5af12 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.763845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:42.783869 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.784473 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.784755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.785031 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.785703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.787657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.808999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1217/2443] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:44:42 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:42.813094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.823564 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955c87a-a5d8-48a4-a382-2e960b317618 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.828188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.828777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.829063 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.829363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.830004 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.831790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.841506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1227/2444] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:42.845246 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.855459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a5f129-f7d5-46c5-87e1-02b86c22960e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.860053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60113039-0b1d-46db-8a35-0ce6d62b1cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.860634 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113039-0b1d-46db-8a35-0ce6d62b1cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.860913 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113039-0b1d-46db-8a35-0ce6d62b1cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.861184 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60113039-0b1d-46db-8a35-0ce6d62b1cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.861843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113039-0b1d-46db-8a35-0ce6d62b1cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.863464 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60113039-0b1d-46db-8a35-0ce6d62b1cf6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.875541 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1218/2445] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:42.879500 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:42.889584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493a56c-694f-4946-b34a-53ec1d3df13f admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:42.894149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:42.894787 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.895068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.895342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] PATH_INFO: `/v3/projects/1c48992900d140cfbae55f52fd439917/users/0401e57284de4fdcace109513142263e/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.909503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c48992900d140cfbae55f52fd439917, user_id=0401e57284de4fdcace109513142263e, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:42.912142 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:42.925607 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1228/2446] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:42 2022] PUT /identity/v3/projects/1c48992900d140cfbae55f52fd439917/users/0401e57284de4fdcace109513142263e/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:42.930682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:42.930967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:42.931291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:42.951004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:42.952200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:43.010366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1219/2447] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:43.020193 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:43.021981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:43.078501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18b9063-57d1-42c6-8707-b7d7da6553d4 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:43.082123 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:43.082719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:43.082994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:43.083260 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:43.093017 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:43.094505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:43.105083 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1229/2448] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:43.659059 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:43.670039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b968e7f2-f112-4a51-aec5-497b731b1963 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:43.674153 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:43.674854 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:43.675197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:43.675574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:43.677436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:43.689773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:43.691192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:43.702549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1220/2449] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:44:44.700666 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:44.702237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:44.893225 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e96249-3f36-4e03-b134-4f14160d2e0c service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:44.900882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:44.901642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:44.902158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:44.902403 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:44.904576 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:44.989870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:45.014804 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:45.016571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db6fdb1-c1e8-4692-bbcd-c911b31e64e5 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:45.028516 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1230/2450] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:44 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:48.965106 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:48.968793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:49.003019 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4480863-14aa-452a-bcb4-fd0d6d640325 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.014229 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.015182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.016165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.017141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.019351 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:49.038999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.041295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.068145 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1221/2451] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:44:49.300416 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.301291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.302279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.340937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] MFA Rules not processed for user `0401e57284de4fdcace109513142263e`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:49.342426 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:49.458276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1231/2452] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:44:49.463235 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:49.505537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0304b-5712-4e37-b70c-08c4c3587fb9 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.515356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.533599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.533978 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.534326 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.535253 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.537275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.580357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1222/2453] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:44:49 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:44:49.581536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:49.598361 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c590ca-9316-4730-966a-d89f1f65169d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.684341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.685149 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.685553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.685922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.686684 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.688880 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.720175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1232/2454] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:44:49 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:44:49.721114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:49.733148 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd17ad3-8c61-4edf-8932-bd7bac13f501 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.738318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.739017 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.739386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.739737 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.740443 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.742174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.754007 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1223/2455] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 12 12:44:49.757930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:49.768624 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c35f268-4c15-46d5-a42e-86bcb78d11d3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.774031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.774817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.775172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.775535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] PATH_INFO: `/v3/projects/e60af9afb0d842ecb7ce0e99b2bf2ea0/users/445f6c7367574b60bc6eed4739d0a07d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.808751 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60af9afb0d842ecb7ce0e99b2bf2ea0, user_id=445f6c7367574b60bc6eed4739d0a07d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.811659 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.830644 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1233/2456] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:49 2022] PUT /identity/v3/projects/e60af9afb0d842ecb7ce0e99b2bf2ea0/users/445f6c7367574b60bc6eed4739d0a07d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:44:49.834671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:49.848128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e7345-a526-4a44-9d87-c65cd2c56a3e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.914967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.915835 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.916119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.916392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.917042 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.918682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.920058 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:49.939069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:49.941079 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1224/2457] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:44:49 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:44:49.950951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b3bc1-a3ac-47fa-8487-4167082702c8 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:49.959053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:49.959736 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:49.960019 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:49.960296 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] PATH_INFO: `/v3/projects/e60af9afb0d842ecb7ce0e99b2bf2ea0/users/445f6c7367574b60bc6eed4739d0a07d/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:49.962386 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60af9afb0d842ecb7ce0e99b2bf2ea0, user_id=445f6c7367574b60bc6eed4739d0a07d, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:49.965747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:49.979278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1234/2458] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:44:49 2022] PUT /identity/v3/projects/e60af9afb0d842ecb7ce0e99b2bf2ea0/users/445f6c7367574b60bc6eed4739d0a07d/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:54.006748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:54.008609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:54.024499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e005fe3-f19a-4e94-9ad3-d106e1f98a75 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:54.122723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:54.123307 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:54.123580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:54.123847 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] PATH_INFO: `/v3/users/4b702145173b471e880425c0703b4d7c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:54.141943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b702145173b471e880425c0703b4d7c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:54.144925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:54.214381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b702145173b471e880425c0703b4d7c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.215456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b702145173b471e880425c0703b4d7c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.216288 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b702145173b471e880425c0703b4d7c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.225299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b702145173b471e880425c0703b4d7c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.246222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b702145173b471e880425c0703b4d7c'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.296171 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1225/2459] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:44:54 2022] DELETE /identity/v3/users/4b702145173b471e880425c0703b4d7c => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:54.570919 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:54.573473 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:54.593542 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc150f0-fbbe-4d9b-8dd3-e1c88f46ce1c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:54.637788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:54.680285 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:54.681675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:54.682031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:54.682378 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] PATH_INFO: `/v3/projects/9e1a9bafab47407599e09fb4d0f8262b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:54.705086 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e1a9bafab47407599e09fb4d0f8262b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:54.707372 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:54.780917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e1a9bafab47407599e09fb4d0f8262b', 'request_id': 'req-d45e499b-f200-4e85-855e-882ec0169f01'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.788600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e1a9bafab47407599e09fb4d0f8262b', 'request_id': 'req-d45e499b-f200-4e85-855e-882ec0169f01'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.799338 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e1a9bafab47407599e09fb4d0f8262b', 'request_id': 'req-d45e499b-f200-4e85-855e-882ec0169f01'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.814874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] The token cache is being invalidate because project 9e1a9bafab47407599e09fb4d0f8262b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:44:54.814874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.815625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:54.817836 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1235/2460] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:44:54 2022] DELETE /identity/v3/projects/9e1a9bafab47407599e09fb4d0f8262b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:54.821394 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:54.840904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb128aee-f19c-4e76-8f4b-b2f26a72976d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:54.926836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:54.927755 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:54.928164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:54.928445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] PATH_INFO: `/v3/users/055839f062d644059a9cf7925cf7f16b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:54.939990 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:54.949694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=055839f062d644059a9cf7925cf7f16b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:54.952816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.047821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '055839f062d644059a9cf7925cf7f16b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.049030 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '055839f062d644059a9cf7925cf7f16b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.049823 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '055839f062d644059a9cf7925cf7f16b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.064258 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '055839f062d644059a9cf7925cf7f16b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.078082 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-a6a309d6-27d9-42c4-b072-1d30f406d5d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '055839f062d644059a9cf7925cf7f16b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.106318 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1226/2461] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:44:54 2022] DELETE /identity/v3/users/055839f062d644059a9cf7925cf7f16b => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:55.330187 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:55.332987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:55.362910 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e499b-f200-4e85-855e-882ec0169f01 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:55.474923 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:55.476005 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:55.476487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:55.476948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] PATH_INFO: `/v3/projects/19d39455ff9d4c8680f344fd18fb072d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:55.496929 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19d39455ff9d4c8680f344fd18fb072d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:55.500122 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:55.599897 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:44:55.606318 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19d39455ff9d4c8680f344fd18fb072d', 'request_id': 'req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.618963 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19d39455ff9d4c8680f344fd18fb072d', 'request_id': 'req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.638299 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19d39455ff9d4c8680f344fd18fb072d', 'request_id': 'req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.671254 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] The token cache is being invalidate because project 19d39455ff9d4c8680f344fd18fb072d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:44:55.672108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.672904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:44:55.689540 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1236/2462] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:44:55 2022] DELETE /identity/v3/projects/19d39455ff9d4c8680f344fd18fb072d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:44:57.197843 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a837-c072-45e1-94b0-fb5fbaf0a2a8 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:57.198793 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a837-c072-45e1-94b0-fb5fbaf0a2a8 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:57.199075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95a837-c072-45e1-94b0-fb5fbaf0a2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:57.228811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-ed95a837-c072-45e1-94b0-fb5fbaf0a2a8 None None] MFA Rules not processed for user `445f6c7367574b60bc6eed4739d0a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:57.230157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-ed95a837-c072-45e1-94b0-fb5fbaf0a2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:57.307260 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1227/2463] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:44:57.323519 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:44:57.324972 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:57.405362 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ae8f2-cb83-4d19-b6e4-aea56c7b2123 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:44:57.411163 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:44:57.411660 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:57.411990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:57.412395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:57.414173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:57.476745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:44:57.478468 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:44:57.487360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1237/2464] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:44:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:44:58.515156 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:44:58.515747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:44:58.515996 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:44:58.537674 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] MFA Rules not processed for user `0401e57284de4fdcace109513142263e`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:44:58.539254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:44:58.548981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1228/2465] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:44:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:45:07.483232 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=4,keystone:SELECT=63,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:08.531814 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=66,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:45:37.471546 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:45:37.473002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:45:37.529084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e579c32e-8408-4f08-b6f1-4de527185eb8 service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:45:37.532757 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:45:37.533382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:45:37.533661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:45:37.533932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:45:37.543178 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:45:37.544611 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:45:37.553883 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1238/2466] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:45:37 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:45:47.549999 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=26 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:12.325932 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:12.327323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:12.383179 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03adf26-6da8-40b0-a5e8-229426509779 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:12.386760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:12.388052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:12.388328 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:12.388601 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:12.390345 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:12.403456 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:12.404878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3484dd00-564d-4acf-8c76-4d3129b27592 service glance] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:12.415551 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1229/2467] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:22.410866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=26 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:24.635602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:24.644773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9cf83-ae83-44e3-a7a3-2528a43dc724 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:24.648341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:24.648928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:24.649201 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:24.649508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:24.651281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:24.665234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:24.666687 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:24.677446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1239/2468] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:29.279199 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.279667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.279779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.299386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:29.300580 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:29.366516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1230/2469] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:46:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:29.425776 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:29.427842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:29.503594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cff3c-19aa-4398-ac7b-20ee8dd384f7 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:29.509517 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c43bb-12b8-4cca-b6cf-47f62775a935 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:29.510237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c43bb-12b8-4cca-b6cf-47f62775a935 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:29.510518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c43bb-12b8-4cca-b6cf-47f62775a935 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:29.510793 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c43bb-12b8-4cca-b6cf-47f62775a935 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:29.522882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c43bb-12b8-4cca-b6cf-47f62775a935 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:29.522882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c43bb-12b8-4cca-b6cf-47f62775a935 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:29.531301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1240/2470] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:46:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:46:39.287611 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:39.289162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:39.300552 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f115c6-9a78-4578-97a3-0addb408c2ab None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:39.304075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:39.304664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:39.304936 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:39.305198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:39.306870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:39.316515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:39.317953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360dad0b-0fee-4ba4-8b74-b7a67424c311 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:39.325779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1231/2471] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:39.526669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=38 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:43.580768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea093a5d-1545-4048-985b-506b7b35f225 None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.581475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea093a5d-1545-4048-985b-506b7b35f225 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.581649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea093a5d-1545-4048-985b-506b7b35f225 None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.583100 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1241/2472] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:46:43.588803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8ab9a-d7aa-4a75-891d-1e186e1e7a70 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.589300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8ab9a-d7aa-4a75-891d-1e186e1e7a70 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.589559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8ab9a-d7aa-4a75-891d-1e186e1e7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.611922 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-c3a8ab9a-d7aa-4a75-891d-1e186e1e7a70 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:43.613170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c3a8ab9a-d7aa-4a75-891d-1e186e1e7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:43.623022 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1232/2473] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:46:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:43.634743 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea093a5d-1545-4048-985b-506b7b35f225 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:43.646199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea093a5d-1545-4048-985b-506b7b35f225 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:43.649681 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f755da-1865-467f-92ff-65f83976ace9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1fb121d7e44f4f868bc28dda9647bbee', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:43.650271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f755da-1865-467f-92ff-65f83976ace9 service glance] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.650544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f755da-1865-467f-92ff-65f83976ace9 service glance] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.650814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f755da-1865-467f-92ff-65f83976ace9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.661822 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f755da-1865-467f-92ff-65f83976ace9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:43.663278 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f755da-1865-467f-92ff-65f83976ace9 service glance] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:43.674788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1242/2474] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:43.744775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17012a-1336-43c7-8fa9-a61806effe55 None None] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.745063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17012a-1336-43c7-8fa9-a61806effe55 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.745366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e17012a-1336-43c7-8fa9-a61806effe55 None None] PATH_INFO: `/` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.746743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1233/2475] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:46:43.751630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c137135-201c-442e-b76d-4eca60286d23 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.751912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c137135-201c-442e-b76d-4eca60286d23 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.752183 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c137135-201c-442e-b76d-4eca60286d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.771544 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-4c137135-201c-442e-b76d-4eca60286d23 None None] MFA Rules not processed for user `afdf63d4c79649a48cddc9f64e10e3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:43.772750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4c137135-201c-442e-b76d-4eca60286d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:43.779739 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1243/2476] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:46:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:43.791137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17012a-1336-43c7-8fa9-a61806effe55 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:43.804151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e17012a-1336-43c7-8fa9-a61806effe55 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:43.810186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8912de06-b8c7-41b7-94b7-4b3c319074bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:43.811037 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912de06-b8c7-41b7-94b7-4b3c319074bb service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.811372 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912de06-b8c7-41b7-94b7-4b3c319074bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.811864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8912de06-b8c7-41b7-94b7-4b3c319074bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.819776 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912de06-b8c7-41b7-94b7-4b3c319074bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:43.821200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8912de06-b8c7-41b7-94b7-4b3c319074bb service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:43.829086 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1234/2477] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:43.912584 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a060-91ec-4c96-90a2-9cc8edb649df None None] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.912874 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a060-91ec-4c96-90a2-9cc8edb649df None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.913154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488a060-91ec-4c96-90a2-9cc8edb649df None None] PATH_INFO: `/` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.914535 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1244/2478] 10.222.0.17 () {54 vars in 921 bytes} [Wed Oct 12 12:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 12 12:46:43.919609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabb32e-0554-4144-ae04-ad545179c435 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.920760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabb32e-0554-4144-ae04-ad545179c435 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.920760 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cabb32e-0554-4144-ae04-ad545179c435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.940313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-8cabb32e-0554-4144-ae04-ad545179c435 None None] MFA Rules not processed for user `5e2c96c17470485f90e95e3e5bb87ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:43.941554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-8cabb32e-0554-4144-ae04-ad545179c435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:43.951871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1235/2479] 10.222.0.17 () {58 vars in 1018 bytes} [Wed Oct 12 12:46:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:43.961530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a060-91ec-4c96-90a2-9cc8edb649df None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:43.973229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488a060-91ec-4c96-90a2-9cc8edb649df None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:43.976716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:43.977292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:43.977597 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:43.977866 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:43.987921 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:43.989351 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:44.000047 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1245/2480] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:46:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:46:45.788780 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb2eed-0f00-4092-ab1d-09a5c4bb9aeb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:45.789271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb2eed-0f00-4092-ab1d-09a5c4bb9aeb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:45.789409 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bb2eed-0f00-4092-ab1d-09a5c4bb9aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:45.808363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-a2bb2eed-0f00-4092-ab1d-09a5c4bb9aeb None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:46:45.809576 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a2bb2eed-0f00-4092-ab1d-09a5c4bb9aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:46:45.816683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1236/2481] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:46:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:46:45.829396 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:46:45.845451 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37890c04-8ab7-480a-b7a7-ad936f9bdabd service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:46:45.850317 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:46:45.850914 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:46:45.851182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:46:45.851450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:46:45.859648 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:46:45.861220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:46:45.869280 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1246/2482] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:46:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:46:55.803062 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=46 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:46:55.865854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=21 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:47:04.956143 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:47:04.956653 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:47:04.956821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:47:04.978096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:47:04.979335 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:47:04.987664 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1237/2483] 10.222.0.17 () {58 vars in 967 bytes} [Wed Oct 12 12:47:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:47:05.001489 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:47:05.012104 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc46db1-05f3-455a-a158-9e1f1b2414ca service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:47:05.017271 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:47:05.017906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:47:05.018191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:47:05.018466 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:47:05.028342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:47:05.030099 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:47:05.038790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1247/2484] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:47:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:47:14.972474 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:47:15.035213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=6 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:08.548232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:08.558275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df48f8-8a27-441a-a00d-5eadc9080220 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:08.562907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:08.563581 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:08.563858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:08.564124 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:08.565880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:08.631553 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:08.633005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:08.643877 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1238/2485] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:11.873966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:11.892869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be86c9a-c7bf-4b52-a587-cb0693e728a7 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:11.896334 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:11.896934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:11.897212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:11.897669 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:11.899613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:11.909908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:11.911335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:11.919614 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1248/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:11 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:11.998440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.000556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.086152 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.097243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24da03a-ffd2-4dd2-a66b-2f7c69dd5c76 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.102258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.103196 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.103530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.104024 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.105926 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.116836 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba9b3e-e641-4dbe-8c8b-7eab361d378f service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.119188 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.120825 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.124575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.125191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.125525 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.125802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.127862 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.129746 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1249/2487] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:48:12.130800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.132125 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.140166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.141757 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.147477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbc79e-0a51-4b74-b156-5e00c4800669 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.152700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1239/2488] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:48:12.153670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.154575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.155678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.156366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] PATH_INFO: `/v3/users/d0010af061ae4490a6855639a6825cac` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.158175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0010af061ae4490a6855639a6825cac)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.160279 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.237856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0010af061ae4490a6855639a6825cac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.246070 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.247832 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0010af061ae4490a6855639a6825cac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.248459 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0010af061ae4490a6855639a6825cac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.256431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0010af061ae4490a6855639a6825cac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.257147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0010af061ae4490a6855639a6825cac'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.270439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecc3b93-8ca7-4bd6-a129-76ae33e69f68 service placement] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.275340 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.275944 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.276228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.276505 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.281282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1250/2489] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:48:12 2022] DELETE /identity/v3/users/d0010af061ae4490a6855639a6825cac => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:12.282578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.298290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.299772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892017c5-933b-4cc7-a109-ce4f648ab693 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.311603 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1240/2490] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:48:12.522952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.524744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.616808 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58ad28-5489-4eb8-b419-10b6028d9340 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.621479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.622012 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.622288 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.622558 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] PATH_INFO: `/v3/projects/e57c943e5713455781e1799934cef6bc` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.623699 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e57c943e5713455781e1799934cef6bc)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.625689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.673115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:12.674649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e57c943e5713455781e1799934cef6bc', 'request_id': 'req-04d0c5f7-2699-490a-8bd9-45cef5838611'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.684000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e57c943e5713455781e1799934cef6bc', 'request_id': 'req-04d0c5f7-2699-490a-8bd9-45cef5838611'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.692837 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e57c943e5713455781e1799934cef6bc', 'request_id': 'req-04d0c5f7-2699-490a-8bd9-45cef5838611'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.704596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] The token cache is being invalidate because project e57c943e5713455781e1799934cef6bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:48:12.704886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.705790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-04d0c5f7-2699-490a-8bd9-45cef5838611 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:12.707282 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1251/2491] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:48:12 2022] DELETE /identity/v3/projects/e57c943e5713455781e1799934cef6bc => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:12.712289 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.712573 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.712844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.732863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:12.734259 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.794483 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1241/2492] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:12.803185 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.803471 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.804148 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.824477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:12.825741 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:12.834675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1252/2493] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:12.838904 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.848668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d44026-d6ca-45bb-b25c-dc1d9acb9048 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.853182 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.853807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.854085 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.854366 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.854997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:48:12.855298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.856880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.863592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:12.869695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1242/2494] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:48:12.873767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.883532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5dc6a7-3a3b-4b0b-9673-a2ecd5a3c40d None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.888182 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.888769 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.889062 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.890212 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.890547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.892482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:12.920165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1253/2495] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:48:12 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:48:12.925194 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:12.935265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb5ac9b-5947-41bb-8e36-0eb1785d9354 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:12.990758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:12.991363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:12.991641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:12.991909 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:12.992527 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:12.994533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:13.015097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1243/2496] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:48:12 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:48:13.019195 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:13.029172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c054c1b0-fe66-4bbc-ad25-b2932015c512 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:13.034337 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:13.034918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:13.035191 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:13.035457 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:13.036082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:13.037698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:13.047912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1254/2497] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:48:13.051613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:13.061567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087a54f-4457-4ee4-b552-381ddc63c2b1 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:13.066720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191ece14-57fd-49c9-b6f4-ac58b2177199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:13.067310 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ece14-57fd-49c9-b6f4-ac58b2177199 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:13.067589 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ece14-57fd-49c9-b6f4-ac58b2177199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:13.067861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191ece14-57fd-49c9-b6f4-ac58b2177199 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:13.068493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ece14-57fd-49c9-b6f4-ac58b2177199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:13.070107 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191ece14-57fd-49c9-b6f4-ac58b2177199 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:13.080549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1244/2498] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:48:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:48:13.084560 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:13.094990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58279c6-abab-4400-8835-6d45d09a447a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:13.099596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:13.100229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:13.100502 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:13.100768 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] PATH_INFO: `/v3/projects/4eb60050b8b041f18702bdf4efce2daa/users/1f2c0c6d332648c7afd15d9052d52599/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:13.116031 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4eb60050b8b041f18702bdf4efce2daa, user_id=1f2c0c6d332648c7afd15d9052d52599, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:13.118748 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:13.131482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1255/2499] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:48:13 2022] PUT /identity/v3/projects/4eb60050b8b041f18702bdf4efce2daa/users/1f2c0c6d332648c7afd15d9052d52599/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:13.142583 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:13.142816 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:13.143088 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:13.172673 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] MFA Rules not processed for user `1f2c0c6d332648c7afd15d9052d52599`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:13.173907 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:13.243263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1245/2500] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:13.255027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:13.256867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:13.359029 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec28d24-5ab6-439f-988f-b4712fd3c612 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:13.365420 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5cc57-6db3-42f9-ae86-2fca4a57c832 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:13.366146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cc57-6db3-42f9-ae86-2fca4a57c832 service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:13.366529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cc57-6db3-42f9-ae86-2fca4a57c832 service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:13.366942 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5cc57-6db3-42f9-ae86-2fca4a57c832 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:13.376712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5cc57-6db3-42f9-ae86-2fca4a57c832 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:13.378637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5cc57-6db3-42f9-ae86-2fca4a57c832 service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:13.388782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1256/2501] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:21.768264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:21.826769 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49683db-e99b-44db-9df9-9c59ca5c595e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:21.832308 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:21.832930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:21.833207 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:21.833512 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] PATH_INFO: `/v3/users/1f2c0c6d332648c7afd15d9052d52599` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:21.834652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f2c0c6d332648c7afd15d9052d52599)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:21.836615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:21.842045 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:21.883574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f2c0c6d332648c7afd15d9052d52599'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:21.884213 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f2c0c6d332648c7afd15d9052d52599'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:21.884807 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f2c0c6d332648c7afd15d9052d52599'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:21.892393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f2c0c6d332648c7afd15d9052d52599'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:21.899584 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f2c0c6d332648c7afd15d9052d52599'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:21.916432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1246/2502] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:48:21 2022] DELETE /identity/v3/users/1f2c0c6d332648c7afd15d9052d52599 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:21.920889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:21.921376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:21.921528 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:21.940694 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:21.941913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:21.990484 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:22.002041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1257/2503] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:22.011781 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.013192 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:22.069628 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc7796-9d9a-481f-bec6-9dbe9fd030e3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.073197 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28404684-6e1c-45bb-916f-df3254355fa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.073815 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28404684-6e1c-45bb-916f-df3254355fa4 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.074089 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28404684-6e1c-45bb-916f-df3254355fa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.074358 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28404684-6e1c-45bb-916f-df3254355fa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.083627 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28404684-6e1c-45bb-916f-df3254355fa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.084995 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28404684-6e1c-45bb-916f-df3254355fa4 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.094282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1247/2504] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:22.371000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.372487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:22.387383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417a89d-3095-45c8-9d98-6d3db117e74e None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.392295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.392924 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.393263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.393575 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] PATH_INFO: `/v3/projects/4eb60050b8b041f18702bdf4efce2daa` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.394696 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4eb60050b8b041f18702bdf4efce2daa)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.396593 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.444621 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4eb60050b8b041f18702bdf4efce2daa', 'request_id': 'req-18c24715-1005-4a11-90cb-02ca7860143a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:22.450749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4eb60050b8b041f18702bdf4efce2daa', 'request_id': 'req-18c24715-1005-4a11-90cb-02ca7860143a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:22.457792 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4eb60050b8b041f18702bdf4efce2daa', 'request_id': 'req-18c24715-1005-4a11-90cb-02ca7860143a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:22.468102 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] The token cache is being invalidate because project 4eb60050b8b041f18702bdf4efce2daa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:48:22.468491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:22.469133 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-18c24715-1005-4a11-90cb-02ca7860143a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:48:22.470760 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1258/2505] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:48:22 2022] DELETE /identity/v3/projects/4eb60050b8b041f18702bdf4efce2daa => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:22.475633 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.475917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.476188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.496388 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:22.497606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:22.556305 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1248/2506] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:22.564367 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.564643 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.564912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.584180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:22.585399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:22.593135 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1259/2507] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:22.597642 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.607063 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2340ebc9-8e24-41fb-8372-17c6a15a7d2e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.612284 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.612875 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.613160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.613466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.614111 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:48:22.614657 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.616244 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.627750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1249/2508] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:48:22.632046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.641589 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37683576-ad20-45d1-8d8e-3201500e193f None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.647689 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.648292 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.648568 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.648842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.649514 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.651445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.678686 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1260/2509] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:48:22 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:48:22.682717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.692558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61420942-8482-4375-baae-41a17b2a224e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.703775 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:22.752135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.752743 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.753020 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.753292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.753958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.755885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.778870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1250/2510] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:48:22 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:48:22.782840 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.793478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41484c1-5da1-460e-8bf4-e8ed322e0fd3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.798915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.799515 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.799818 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.800094 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.800800 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.802427 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.813209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1261/2511] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:48:22.817026 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.827313 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a21925-8e42-46df-b502-e6d86bc5b59f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.832281 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aa8d7a-0c5f-487f-8990-24dd0939881d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.832910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8d7a-0c5f-487f-8990-24dd0939881d admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.833189 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8d7a-0c5f-487f-8990-24dd0939881d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.833494 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aa8d7a-0c5f-487f-8990-24dd0939881d admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.834166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa8d7a-0c5f-487f-8990-24dd0939881d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.835761 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aa8d7a-0c5f-487f-8990-24dd0939881d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.847271 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1251/2512] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:48:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:48:22.851414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:22.861997 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ad59-c715-4a17-ab7e-5e1a6ded70d7 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:22.866892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:22.867537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.867816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.868088 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] PATH_INFO: `/v3/projects/9487fc3c9c4b4406aa483d3803e8c3df/users/e21f85d86a644fdf9c27a01f41a3ba6f/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.883177 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9487fc3c9c4b4406aa483d3803e8c3df, user_id=e21f85d86a644fdf9c27a01f41a3ba6f, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:22.885864 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:22.898991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1262/2513] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:48:22 2022] PUT /identity/v3/projects/9487fc3c9c4b4406aa483d3803e8c3df/users/e21f85d86a644fdf9c27a01f41a3ba6f/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:22.910065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:22.910346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:22.910618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:22.939383 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] MFA Rules not processed for user `e21f85d86a644fdf9c27a01f41a3ba6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:22.940574 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:23.003923 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1252/2514] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:48:23.008383 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.071711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c127-6472-49fe-ba19-1308e1e3223a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.076343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.076936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.077217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.077522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.078632 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.080603 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.106596 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1263/2515] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:48:23 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:48:23.110625 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.121476 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e83ae-3f7c-4b73-a27c-f02e2b9fb137 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.176395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.176999 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.177337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.177668 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.178321 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.180221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.201139 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1253/2516] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:48:23 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:48:23.205917 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.211385 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:23.217294 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da2861-bba4-4217-a3e0-e6940e4401df admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.223243 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.223839 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.224118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.224390 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.225056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.226703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.237057 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1264/2517] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:48:23.241384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.263142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad3948-4da7-4798-9883-d3c93deefa8d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.268636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.269417 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.269702 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.269977 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] PATH_INFO: `/v3/projects/bd97239fa96c450fa29982bf6d0db6e4/users/396cf17b25f54a55b9f6d600e95c699b/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.282123 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:23.286770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd97239fa96c450fa29982bf6d0db6e4, user_id=396cf17b25f54a55b9f6d600e95c699b, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.289590 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.303028 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1254/2518] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:48:23 2022] PUT /identity/v3/projects/bd97239fa96c450fa29982bf6d0db6e4/users/396cf17b25f54a55b9f6d600e95c699b/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:23.307175 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.317608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef21998c-0f0d-4e87-b6bc-1adf27842d9a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.372491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ce480-1d97-4a4f-ba44-cce3cded28d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.373028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce480-1d97-4a4f-ba44-cce3cded28d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.373342 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce480-1d97-4a4f-ba44-cce3cded28d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.373613 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ce480-1d97-4a4f-ba44-cce3cded28d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.374251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ce480-1d97-4a4f-ba44-cce3cded28d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.376347 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ce480-1d97-4a4f-ba44-cce3cded28d6 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.386034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1265/2519] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:48:23.390127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.400868 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17b58db-cdb5-4b8e-986a-43347732049d admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.405413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.406053 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.406384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.406652 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] PATH_INFO: `/v3/projects/bd97239fa96c450fa29982bf6d0db6e4/users/396cf17b25f54a55b9f6d600e95c699b/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.408560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd97239fa96c450fa29982bf6d0db6e4, user_id=396cf17b25f54a55b9f6d600e95c699b, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.411218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.421134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1255/2520] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:48:23 2022] PUT /identity/v3/projects/bd97239fa96c450fa29982bf6d0db6e4/users/396cf17b25f54a55b9f6d600e95c699b/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:48:23.483164 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.483445 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.483713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.511933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] MFA Rules not processed for user `396cf17b25f54a55b9f6d600e95c699b`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:48:23.513141 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:23.580291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1266/2521] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:48:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:48:23.591238 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.592950 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:48:23.651453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509abbaf-e0ab-47ba-9679-8a0485db4a63 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.654973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.655566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.655844 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.656116 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.665299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.666770 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.676510 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1256/2522] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:23.752995 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:48:23.762617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a8693-c16f-4e1d-af13-64e59e8f7027 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:48:23.766208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684b2994-007f-4c72-a32e-d26186da0440 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:48:23.766791 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b2994-007f-4c72-a32e-d26186da0440 service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:48:23.767069 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b2994-007f-4c72-a32e-d26186da0440 service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:48:23.767344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684b2994-007f-4c72-a32e-d26186da0440 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:48:23.824246 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b2994-007f-4c72-a32e-d26186da0440 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:48:23.825718 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684b2994-007f-4c72-a32e-d26186da0440 service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:48:23.835075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1267/2523] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:48:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:48:33.671892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=35,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:48:33.830656 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:06.238113 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:06.301046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5870-26dd-47e3-a54b-6c7b66ff2a1b service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:06.305550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:06.306165 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:06.306459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:06.306727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] PATH_INFO: `/v3/users/e21f85d86a644fdf9c27a01f41a3ba6f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:06.307828 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e21f85d86a644fdf9c27a01f41a3ba6f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:06.309764 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:06.354705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e21f85d86a644fdf9c27a01f41a3ba6f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.355368 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e21f85d86a644fdf9c27a01f41a3ba6f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.356002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e21f85d86a644fdf9c27a01f41a3ba6f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.363408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e21f85d86a644fdf9c27a01f41a3ba6f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.370952 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e21f85d86a644fdf9c27a01f41a3ba6f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.389081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1257/2524] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:06 2022] DELETE /identity/v3/users/e21f85d86a644fdf9c27a01f41a3ba6f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:06.393435 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:06.393906 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:06.394026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:06.414765 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:06.415966 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:06.474344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1268/2525] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:06.483873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:06.485384 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:06.542175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44839c1a-f44a-4a6f-a7b6-8dfe649e0d26 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:06.545851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:06.546445 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:06.546722 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:06.546991 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:06.556569 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:06.558018 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:06.567819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1258/2526] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:06.832940 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:06.834399 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:06.845961 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f94a63-c599-4377-b9ea-1a5e653636ee None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:06.852716 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:06.853645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:06.853991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:06.854324 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] PATH_INFO: `/v3/projects/9487fc3c9c4b4406aa483d3803e8c3df` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:06.855704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9487fc3c9c4b4406aa483d3803e8c3df)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:06.858414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:06.907341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9487fc3c9c4b4406aa483d3803e8c3df', 'request_id': 'req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.914291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9487fc3c9c4b4406aa483d3803e8c3df', 'request_id': 'req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.921227 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9487fc3c9c4b4406aa483d3803e8c3df', 'request_id': 'req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.932353 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] The token cache is being invalidate because project 9487fc3c9c4b4406aa483d3803e8c3df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:06.932640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.933286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:06.934784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1269/2527] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:06 2022] DELETE /identity/v3/projects/9487fc3c9c4b4406aa483d3803e8c3df => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:06.938750 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:06.948845 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99835373-480c-45a0-a264-65dd4ae4d362 service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:06.966921 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:07.007254 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.007873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.008152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.008424 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] PATH_INFO: `/v3/users/396cf17b25f54a55b9f6d600e95c699b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.009679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=396cf17b25f54a55b9f6d600e95c699b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.011615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.055878 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '396cf17b25f54a55b9f6d600e95c699b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.056518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '396cf17b25f54a55b9f6d600e95c699b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.057117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '396cf17b25f54a55b9f6d600e95c699b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.063873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '396cf17b25f54a55b9f6d600e95c699b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.070992 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-6e65940b-a9d8-4f6b-a154-03c8478e0aa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '396cf17b25f54a55b9f6d600e95c699b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.088459 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1259/2528] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:06 2022] DELETE /identity/v3/users/396cf17b25f54a55b9f6d600e95c699b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:07.360465 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.362042 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:07.375698 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f562b-9e6c-4ece-82e3-10e3b4af02bf admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.432400 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.433106 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.433297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.433602 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] PATH_INFO: `/v3/projects/bd97239fa96c450fa29982bf6d0db6e4` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.434729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd97239fa96c450fa29982bf6d0db6e4)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.436633 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.466706 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:07.484536 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd97239fa96c450fa29982bf6d0db6e4', 'request_id': 'req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.491590 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd97239fa96c450fa29982bf6d0db6e4', 'request_id': 'req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.499091 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd97239fa96c450fa29982bf6d0db6e4', 'request_id': 'req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.510433 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] The token cache is being invalidate because project bd97239fa96c450fa29982bf6d0db6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81424) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:07.510717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.511333 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-cdc56a17-b511-4704-a8d4-f8ceb4536b6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:07.512782 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1270/2529] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:07 2022] DELETE /identity/v3/projects/bd97239fa96c450fa29982bf6d0db6e4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:07.517860 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.518324 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.518440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.538858 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:07.540046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:07.598694 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1260/2530] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:07.607242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.607527 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.607801 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.629132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:07.630409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:07.638425 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1271/2531] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:07.642369 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.652819 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48170918-d158-438a-b0a6-4a930c14bb5f None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.657475 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.658096 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.658396 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.658747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.659338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81425) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:49:07.659641 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.661422 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.672795 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1261/2532] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:49:07.677056 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.686165 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651e9f3-aa26-4fae-bb7b-b1e7aabd6642 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.690692 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.691788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.692194 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.692476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.693097 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.695032 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.721990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1272/2533] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:07 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:07.726021 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.735826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e313f5-8fe0-4955-b4b4-35ea6d0c9d92 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.762371 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:07.794839 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.795435 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.795713 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.796039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.796665 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.798667 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.820596 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1262/2534] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:07 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:07.824711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.835261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf94de8a-7347-4fc2-9e8a-bd9c4c9ffb81 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.839784 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.840370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.840645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.840915 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.841586 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.843170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.853835 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1273/2535] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:07.857566 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.868061 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4332277-2257-4a48-aa73-536fd656dac9 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.872615 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4498b0-817f-48b9-b779-e9e4847cced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.873191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4498b0-817f-48b9-b779-e9e4847cced6 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.873503 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4498b0-817f-48b9-b779-e9e4847cced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.873774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4498b0-817f-48b9-b779-e9e4847cced6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.874420 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4498b0-817f-48b9-b779-e9e4847cced6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.875997 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4498b0-817f-48b9-b779-e9e4847cced6 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.887187 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1263/2536] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:07.891242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:07.901476 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196c7d49-c102-4513-9b40-268db96ee811 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:07.906071 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:07.906709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.906988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.907261 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] PATH_INFO: `/v3/projects/71270d1aeb4044c18e81c8b85bf2a0f3/users/be7da02480234cdaa0dbe4c4214a750d/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.921637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71270d1aeb4044c18e81c8b85bf2a0f3, user_id=be7da02480234cdaa0dbe4c4214a750d, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:07.924263 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:07.937144 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1274/2537] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:07 2022] PUT /identity/v3/projects/71270d1aeb4044c18e81c8b85bf2a0f3/users/be7da02480234cdaa0dbe4c4214a750d/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:49:07.948100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:07.948389 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:07.948659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:07.975700 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] MFA Rules not processed for user `be7da02480234cdaa0dbe4c4214a750d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:07.976886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:08.041047 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1264/2538] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:08.045673 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.106722 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf873d3-2d72-4e6d-98bb-0c2cc8f1a7c2 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.111293 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.111882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.112160 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.112430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.113968 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.116052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.141918 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1275/2539] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:08 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:08.146051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.156954 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745a46bf-78b6-4c10-90cd-5b4f1711ee7e None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.212355 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.212876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.213151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.213453 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.214074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.215973 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.237137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1265/2540] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:08 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:08.242189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.252363 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9355d6-ad2c-472d-bc26-17352bb30d2a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.257207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.258060 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.258343 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.258617 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.259262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.260862 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.271729 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1276/2541] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:08.276076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.286526 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae78c01-eac1-45c8-bf74-75c943f9bfd3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.291052 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.291682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.291958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.292228 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] PATH_INFO: `/v3/projects/eb98eb9bac7d4a08897b89b557648305/users/d925db39157a4a7f888484979b2bbff3/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.307219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb98eb9bac7d4a08897b89b557648305, user_id=d925db39157a4a7f888484979b2bbff3, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.309984 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.322871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1266/2542] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:08 2022] PUT /identity/v3/projects/eb98eb9bac7d4a08897b89b557648305/users/d925db39157a4a7f888484979b2bbff3/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:08.326877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.336733 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57587f-73e2-44cf-b223-5559dad6d138 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.356745 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:08.399803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d17c553-dda9-4978-b144-b34d3b810fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.400412 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17c553-dda9-4978-b144-b34d3b810fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.400723 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17c553-dda9-4978-b144-b34d3b810fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.401000 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d17c553-dda9-4978-b144-b34d3b810fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.401668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17c553-dda9-4978-b144-b34d3b810fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.403262 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d17c553-dda9-4978-b144-b34d3b810fb0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.413233 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1277/2543] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:08.417466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.422915 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:08.429953 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a83dfd-8779-4250-9d4b-9193b2042b4b admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.434963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.435605 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.435879 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.436169 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] PATH_INFO: `/v3/projects/eb98eb9bac7d4a08897b89b557648305/users/d925db39157a4a7f888484979b2bbff3/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.438097 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb98eb9bac7d4a08897b89b557648305, user_id=d925db39157a4a7f888484979b2bbff3, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.440752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.451023 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1267/2544] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:08 2022] PUT /identity/v3/projects/eb98eb9bac7d4a08897b89b557648305/users/d925db39157a4a7f888484979b2bbff3/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:08.461146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.461705 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.461823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.490735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] MFA Rules not processed for user `d925db39157a4a7f888484979b2bbff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:08.491934 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:08.557487 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1278/2545] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:08.568870 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:08.570623 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:08.627650 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f311a6-d9ba-4c61-b53b-f306d2f067ab admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:08.631349 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa49d28-a7a0-46d4-a4f6-c8f325f5c0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:08.631938 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49d28-a7a0-46d4-a4f6-c8f325f5c0cd service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:08.632248 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49d28-a7a0-46d4-a4f6-c8f325f5c0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:08.632520 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49d28-a7a0-46d4-a4f6-c8f325f5c0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:08.641500 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa49d28-a7a0-46d4-a4f6-c8f325f5c0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:08.642930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa49d28-a7a0-46d4-a4f6-c8f325f5c0cd service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:08.652884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1268/2546] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:09.031843 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:09.111957 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1dbca-a2c3-4b9a-9ac8-41f91a7c2304 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:09.117158 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:09.117803 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:09.118076 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:09.118341 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] PATH_INFO: `/v3/users/be7da02480234cdaa0dbe4c4214a750d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:09.119478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=be7da02480234cdaa0dbe4c4214a750d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:09.121402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:09.150664 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:09.170273 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be7da02480234cdaa0dbe4c4214a750d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.177553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be7da02480234cdaa0dbe4c4214a750d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.178255 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be7da02480234cdaa0dbe4c4214a750d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.184987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be7da02480234cdaa0dbe4c4214a750d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.185626 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be7da02480234cdaa0dbe4c4214a750d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.202668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1279/2547] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:09 2022] DELETE /identity/v3/users/be7da02480234cdaa0dbe4c4214a750d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:09.207753 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:09.208039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:09.208323 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:09.236247 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:09.237491 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:09.297221 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1269/2548] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:49:09.307556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:09.309001 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:09.365439 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461c22f-9d60-4eda-917a-647f58a0f7ee admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:09.369026 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:09.369661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:09.369941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:09.370213 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:09.379591 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:09.381027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:09.390479 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1280/2549] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:09.654329 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:09.655951 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:09.674039 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb589e-248e-4662-bf01-0e9535979f97 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:09.679748 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:09.680325 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:09.680599 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:09.680864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] PATH_INFO: `/v3/projects/71270d1aeb4044c18e81c8b85bf2a0f3` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:09.681985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71270d1aeb4044c18e81c8b85bf2a0f3)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:09.683885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:09.730744 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71270d1aeb4044c18e81c8b85bf2a0f3', 'request_id': 'req-b676a093-7fca-4014-bc81-2e8b517ec022'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.739117 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71270d1aeb4044c18e81c8b85bf2a0f3', 'request_id': 'req-b676a093-7fca-4014-bc81-2e8b517ec022'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.745516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71270d1aeb4044c18e81c8b85bf2a0f3', 'request_id': 'req-b676a093-7fca-4014-bc81-2e8b517ec022'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.755759 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] The token cache is being invalidate because project 71270d1aeb4044c18e81c8b85bf2a0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:09.756051 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.756684 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.758196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1270/2550] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:09 2022] DELETE /identity/v3/projects/71270d1aeb4044c18e81c8b85bf2a0f3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:09.762173 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:09.771747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8661b-dedf-40f2-a7eb-634a12d79bc6 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:09.827852 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:09.828408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:09.828670 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:09.828944 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] PATH_INFO: `/v3/users/d925db39157a4a7f888484979b2bbff3` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:09.830090 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] RBAC: Authorizing `identity:delete_user(user_id=d925db39157a4a7f888484979b2bbff3)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:09.832170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:09.879452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd925db39157a4a7f888484979b2bbff3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.881935 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:09.887189 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd925db39157a4a7f888484979b2bbff3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.887984 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd925db39157a4a7f888484979b2bbff3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.895895 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd925db39157a4a7f888484979b2bbff3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.896530 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e1069151-f3fc-4c2d-bb32-4a4e3fd51fef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd925db39157a4a7f888484979b2bbff3'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:09.913996 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1281/2551] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:09 2022] DELETE /identity/v3/users/d925db39157a4a7f888484979b2bbff3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:10.184430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.185888 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:10.198595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676a093-7fca-4014-bc81-2e8b517ec022 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.202214 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:10.258613 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.259255 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.259533 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.259811 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] PATH_INFO: `/v3/projects/eb98eb9bac7d4a08897b89b557648305` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.260934 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb98eb9bac7d4a08897b89b557648305)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.262876 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.310168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb98eb9bac7d4a08897b89b557648305', 'request_id': 'req-2df79fa5-8fe3-4237-a7fa-7363d26f9001'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:10.317785 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb98eb9bac7d4a08897b89b557648305', 'request_id': 'req-2df79fa5-8fe3-4237-a7fa-7363d26f9001'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:10.324232 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb98eb9bac7d4a08897b89b557648305', 'request_id': 'req-2df79fa5-8fe3-4237-a7fa-7363d26f9001'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:10.334981 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] The token cache is being invalidate because project eb98eb9bac7d4a08897b89b557648305 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:10.335267 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:10.335895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-2df79fa5-8fe3-4237-a7fa-7363d26f9001 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:10.337380 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1271/2552] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:10 2022] DELETE /identity/v3/projects/eb98eb9bac7d4a08897b89b557648305 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:10.342207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.342496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.342780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.362585 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:10.364080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:10.422869 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1282/2553] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:10.430620 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.430906 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.431175 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.451087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:10.452273 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:10.460789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1272/2554] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:10.464912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.474483 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fdf618-e689-41c7-91d1-a3ba8268cd99 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.479034 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.479625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.479902 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.480171 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.480810 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:49:10.481110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.482749 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.494308 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1283/2555] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:49:10.498423 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.508076 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3172fe-6c01-4915-b98c-17f4c0477a8d None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.512725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.513322 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.513618 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.513889 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.514518 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.516407 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.543465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1273/2556] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/projects => generated 407 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:49:10.547731 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.557886 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99334be-96df-4fcc-b28a-224e9e1e1a7a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.612767 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.613411 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.613941 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.614207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.614871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.616799 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.637503 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1284/2557] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/users => generated 304 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:10.641749 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.652549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c0629c-7038-4db4-afe8-f2591f09cdd0 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.657087 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.657704 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.657966 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.658237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.658871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.660658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.671558 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1274/2558] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:10.675992 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.685859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf119e0-b42c-43a2-ba6f-742c4634d6e3 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.690401 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78601b81-f482-4f7c-b482-b3e62ba99c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.691574 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601b81-f482-4f7c-b482-b3e62ba99c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.691865 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601b81-f482-4f7c-b482-b3e62ba99c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.692217 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78601b81-f482-4f7c-b482-b3e62ba99c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.692856 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601b81-f482-4f7c-b482-b3e62ba99c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.694482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78601b81-f482-4f7c-b482-b3e62ba99c67 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.704819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1285/2559] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:10.709564 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.719269 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a16804-446d-4be8-944d-df2c200dcbf8 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.723905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.724531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.724810 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.725078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] PATH_INFO: `/v3/projects/679e5442803748f7a82bbdba51ed382e/users/91531b5bdc3f415cb1447b0982419d64/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.740597 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=679e5442803748f7a82bbdba51ed382e, user_id=91531b5bdc3f415cb1447b0982419d64, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.743265 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.757166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1275/2560] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:10 2022] PUT /identity/v3/projects/679e5442803748f7a82bbdba51ed382e/users/91531b5bdc3f415cb1447b0982419d64/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:49:10.767477 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.767962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.768140 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.796889 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] MFA Rules not processed for user `91531b5bdc3f415cb1447b0982419d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:10.798115 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:10.819522 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:10.865449 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1286/2561] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:10.869861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.904142 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:10.932430 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292e436-9e79-4649-a84a-36b0f0d91a4c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:10.936985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:10.937604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:10.937882 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:10.938150 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:10.938766 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:10.940647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:10.966168 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1276/2562] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:10.970335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:10.981709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec41419-d706-4aec-a8f0-1cbbb97db9cc None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.038184 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.038778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.039054 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.039320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.039943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.041877 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.062662 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1287/2563] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:10 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:11.067363 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:11.077588 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194beafe-e189-43b4-b442-31e4763ff5bb admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.082276 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.082859 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.083141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.083415 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.084046 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.085677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.095993 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1277/2564] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:11.099933 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:11.109649 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9edec9e-471b-423a-b620-2be3a830dd24 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.115533 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.116108 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.116382 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.116651 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] PATH_INFO: `/v3/projects/5e0b31c461694909bfd371474cf11b5c/users/3f5ea74845a34ee4a8a51e685e60568a/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.131981 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0b31c461694909bfd371474cf11b5c, user_id=3f5ea74845a34ee4a8a51e685e60568a, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.134650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.147550 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1288/2565] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:11 2022] PUT /identity/v3/projects/5e0b31c461694909bfd371474cf11b5c/users/3f5ea74845a34ee4a8a51e685e60568a/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:11.151725 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:11.163065 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df379289-7761-4e82-a074-658646ff1916 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.217132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b075ff-5ca4-4a33-815a-821367aec54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.217756 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b075ff-5ca4-4a33-815a-821367aec54c admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.218029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b075ff-5ca4-4a33-815a-821367aec54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.218292 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b075ff-5ca4-4a33-815a-821367aec54c admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.218910 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b075ff-5ca4-4a33-815a-821367aec54c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.220488 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b075ff-5ca4-4a33-815a-821367aec54c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.230942 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1278/2566] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:11.235118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:11.246082 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757520ca-a0ae-401c-9204-f4c266af8d85 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.250594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.251234 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.251511 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.251780 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] PATH_INFO: `/v3/projects/5e0b31c461694909bfd371474cf11b5c/users/3f5ea74845a34ee4a8a51e685e60568a/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.253675 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0b31c461694909bfd371474cf11b5c, user_id=3f5ea74845a34ee4a8a51e685e60568a, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.256326 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.266650 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1289/2567] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:11 2022] PUT /identity/v3/projects/5e0b31c461694909bfd371474cf11b5c/users/3f5ea74845a34ee4a8a51e685e60568a/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:11.278162 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.278446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.278717 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.307218 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] MFA Rules not processed for user `3f5ea74845a34ee4a8a51e685e60568a`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:11.308408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:11.375452 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1279/2568] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:11 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:11.386146 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:11.387956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:11.445002 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d588f-ef40-4cf1-b00a-7534a07da0b9 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.448645 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.449235 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.449551 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.449821 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.458833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.460238 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.469422 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1290/2569] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:11 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:11.621219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:11.630740 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a0cdb-c151-4daf-89af-29ac58840820 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:11.634385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b031c-9306-4319-aa25-228b0e9b7561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:11.634985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b031c-9306-4319-aa25-228b0e9b7561 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:11.635263 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b031c-9306-4319-aa25-228b0e9b7561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:11.635537 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b031c-9306-4319-aa25-228b0e9b7561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:11.662545 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:11.692302 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b031c-9306-4319-aa25-228b0e9b7561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:11.693765 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b031c-9306-4319-aa25-228b0e9b7561 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:11.702493 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1280/2570] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:11 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:20.031380 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:20.051087 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:20.096506 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b50dfc8-7913-4b24-be0e-bd6a1e2eb18b service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:20.101954 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:20.102277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:20.102553 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:20.102826 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] PATH_INFO: `/v3/users/91531b5bdc3f415cb1447b0982419d64` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:20.103947 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91531b5bdc3f415cb1447b0982419d64)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:20.105898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:20.153020 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91531b5bdc3f415cb1447b0982419d64'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.162395 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91531b5bdc3f415cb1447b0982419d64'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.163075 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91531b5bdc3f415cb1447b0982419d64'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.170319 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91531b5bdc3f415cb1447b0982419d64'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.170937 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91531b5bdc3f415cb1447b0982419d64'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.192894 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1291/2571] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:20 2022] DELETE /identity/v3/users/91531b5bdc3f415cb1447b0982419d64 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:20.198283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:20.198779 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:20.199264 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:20.220201 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:20.221557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:20.285514 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1281/2572] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:20.295297 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:20.296754 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:20.352778 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0c530e-83df-4c94-9c7f-1db9aeb42e84 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:20.356344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:20.356962 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:20.357237 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:20.357532 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:20.366569 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:20.368095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:20.377259 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1292/2573] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:20.644598 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:20.646164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:20.658200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ccc31-4e9d-4e48-9f48-51a78f400ed4 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:20.672485 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:20.673141 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:20.673482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:20.673732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] PATH_INFO: `/v3/projects/679e5442803748f7a82bbdba51ed382e` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:20.674911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=679e5442803748f7a82bbdba51ed382e)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:20.677290 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:20.723901 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '679e5442803748f7a82bbdba51ed382e', 'request_id': 'req-847d2e84-985e-4842-921d-cb0ff272c9ec'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.730931 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '679e5442803748f7a82bbdba51ed382e', 'request_id': 'req-847d2e84-985e-4842-921d-cb0ff272c9ec'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.737223 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '679e5442803748f7a82bbdba51ed382e', 'request_id': 'req-847d2e84-985e-4842-921d-cb0ff272c9ec'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.748357 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] The token cache is being invalidate because project 679e5442803748f7a82bbdba51ed382e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:20.748645 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.749352 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.750856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1282/2574] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:20 2022] DELETE /identity/v3/projects/679e5442803748f7a82bbdba51ed382e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:20.755849 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:20.765452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e6e322-3ac4-4521-8823-1c2afa6bda7f service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:20.804330 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:20.825155 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:20.825948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:20.826230 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:20.826501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] PATH_INFO: `/v3/users/3f5ea74845a34ee4a8a51e685e60568a` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:20.827604 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f5ea74845a34ee4a8a51e685e60568a)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:20.829562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:20.876229 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f5ea74845a34ee4a8a51e685e60568a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.886095 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f5ea74845a34ee4a8a51e685e60568a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.886717 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f5ea74845a34ee4a8a51e685e60568a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.894491 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f5ea74845a34ee4a8a51e685e60568a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.895180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-1818f987-f5f8-4c32-91df-adaedae91da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f5ea74845a34ee4a8a51e685e60568a'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:20.912842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1293/2575] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:20 2022] DELETE /identity/v3/users/3f5ea74845a34ee4a8a51e685e60568a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:21.177609 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.179108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:21.195703 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d2e84-985e-4842-921d-cb0ff272c9ec admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.239402 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:21.252958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.254285 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.254561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.254831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] PATH_INFO: `/v3/projects/5e0b31c461694909bfd371474cf11b5c` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.255974 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e0b31c461694909bfd371474cf11b5c)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.258004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.303867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e0b31c461694909bfd371474cf11b5c', 'request_id': 'req-15e0e863-d227-47e4-9917-c7214344f047'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:21.311099 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e0b31c461694909bfd371474cf11b5c', 'request_id': 'req-15e0e863-d227-47e4-9917-c7214344f047'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:21.319813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e0b31c461694909bfd371474cf11b5c', 'request_id': 'req-15e0e863-d227-47e4-9917-c7214344f047'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:21.330127 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] The token cache is being invalidate because project 5e0b31c461694909bfd371474cf11b5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:21.330436 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:21.331080 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-15e0e863-d227-47e4-9917-c7214344f047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:21.332683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1283/2576] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:21 2022] DELETE /identity/v3/projects/5e0b31c461694909bfd371474cf11b5c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:21.337571 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.338068 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.338226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.357512 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:21.358703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:21.416453 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1294/2577] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:21.424291 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.424577 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.424880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.445222 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:21.446432 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:21.454251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1284/2578] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:21.458331 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.468038 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a366928c-3a03-49e2-9887-d9aae36b4969 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.473084 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.473712 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.473994 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.474266 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.474907 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:49:21.475208 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.476823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.489064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1295/2579] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 12 12:49:21.493450 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.502392 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3600dfa-c431-4cf7-85a7-2d300a21cc82 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.507861 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.508499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.508720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.508989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.509666 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.511540 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.537673 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1285/2580] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:21.541668 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.552281 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871cdfa7-5a24-42a1-a549-628e5f5b0947 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.595172 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:21.609190 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.609817 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.610092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.610360 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.610988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.613859 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.635040 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1296/2581] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:49:21.639108 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.649275 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ac6fb7-fbd5-441a-8f2d-eda64761e3c3 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.653804 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.654387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.654663 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.654930 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.655554 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.657122 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.667821 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1286/2582] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:21.671637 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.682052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc1e2a-aa9c-4232-b553-bb72384a9c1a admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.686638 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18db6c7e-cae3-418a-957a-16212eea38cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.687223 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6c7e-cae3-418a-957a-16212eea38cf admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.687501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6c7e-cae3-418a-957a-16212eea38cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.687773 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18db6c7e-cae3-418a-957a-16212eea38cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.688408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6c7e-cae3-418a-957a-16212eea38cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.690033 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18db6c7e-cae3-418a-957a-16212eea38cf admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.700408 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1297/2583] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:21.704395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.714190 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29b9e4d-bdbb-463c-8705-e71bb9b32ed4 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.718738 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.719365 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.719638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.719905 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] PATH_INFO: `/v3/projects/c20cc2b6d83242faa40568b217cf143d/users/3c105a12b12543bf89a681e48e68b469/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.734462 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c20cc2b6d83242faa40568b217cf143d, user_id=3c105a12b12543bf89a681e48e68b469, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.737072 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.750638 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1287/2584] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:21 2022] PUT /identity/v3/projects/c20cc2b6d83242faa40568b217cf143d/users/3c105a12b12543bf89a681e48e68b469/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:21.760928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.761161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.761463 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.789756 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] MFA Rules not processed for user `3c105a12b12543bf89a681e48e68b469`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:21.790956 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:21.855490 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1298/2585] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:21.859976 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.919610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb628fd-6c49-416d-aef7-5aef7c793a21 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:21.924240 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:21.924831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:21.925732 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:21.926005 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:21.926626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:21.928509 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:21.947853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:21.955975 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1288/2586] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:21.959965 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:21.971079 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b879d6e-dfb3-4fbc-9f18-e1a2b2e3ba26 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.026988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.027521 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.027802 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.028078 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.028715 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.030678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.053379 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1299/2587] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:21 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:22.057373 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.067797 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21d2de-dd9d-47c3-aadb-0b0cf2f8a97c admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.072416 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.073002 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.073283 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.073578 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.074212 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.075802 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.086640 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1289/2588] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:22.091738 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.102598 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3dd1d2-1d98-4aa8-a538-aa0df6790a3e admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.107226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.107858 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.108132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.108402 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] PATH_INFO: `/v3/projects/5153942312b3478883fa2ef82390ec43/users/53c1f60ae7ff449ababc87af4c5fb1f7/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.123247 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5153942312b3478883fa2ef82390ec43, user_id=53c1f60ae7ff449ababc87af4c5fb1f7, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.125932 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.138842 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1300/2589] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:22 2022] PUT /identity/v3/projects/5153942312b3478883fa2ef82390ec43/users/53c1f60ae7ff449ababc87af4c5fb1f7/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:49:22.142727 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.153135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122361a5-32da-4b0a-819e-05f7e2a07675 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.209961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaaf279-b862-481b-a4e7-7cc6c8e7f837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.210561 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaf279-b862-481b-a4e7-7cc6c8e7f837 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.210841 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaf279-b862-481b-a4e7-7cc6c8e7f837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.211114 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaaf279-b862-481b-a4e7-7cc6c8e7f837 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.211772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaf279-b862-481b-a4e7-7cc6c8e7f837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.213386 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaaf279-b862-481b-a4e7-7cc6c8e7f837 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.223557 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1290/2590] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:22.227854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.238655 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a03c18-0cff-429c-b50f-440b98777e98 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.243286 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.243928 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.244205 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.244482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] PATH_INFO: `/v3/projects/5153942312b3478883fa2ef82390ec43/users/53c1f60ae7ff449ababc87af4c5fb1f7/roles/2e857246137f4d06b40cdabedf42881f` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.246377 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5153942312b3478883fa2ef82390ec43, user_id=53c1f60ae7ff449ababc87af4c5fb1f7, role_id=2e857246137f4d06b40cdabedf42881f)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.249030 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.262693 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1301/2591] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:22 2022] PUT /identity/v3/projects/5153942312b3478883fa2ef82390ec43/users/53c1f60ae7ff449ababc87af4c5fb1f7/roles/2e857246137f4d06b40cdabedf42881f => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:22.274592 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.275215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.275215 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.303885 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] MFA Rules not processed for user `53c1f60ae7ff449ababc87af4c5fb1f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:22.305100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:22.372556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1291/2592] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:22 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:22.383952 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.385728 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:22.395998 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:22.463601 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad873f6-2f96-42bc-8a3c-1148a3575d76 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.467295 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.467898 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.468204 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.468475 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.477423 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.478881 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.488460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1302/2593] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:22.561066 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.571692 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d1524-d9b3-46c7-9a56-6bce406e8b99 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.575338 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109cf95-bc07-4e59-bbf3-677e636106ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.575958 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109cf95-bc07-4e59-bbf3-677e636106ee service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.576237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109cf95-bc07-4e59-bbf3-677e636106ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.576528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109cf95-bc07-4e59-bbf3-677e636106ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.632385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109cf95-bc07-4e59-bbf3-677e636106ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.633886 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0109cf95-bc07-4e59-bbf3-677e636106ee service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:22.642831 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1292/2594] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:22.901431 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:22.961460 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991952bc-540a-4d43-b443-8030bf105e8e service cinder] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:22.965922 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:22.966537 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:22.966816 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:22.967089 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] PATH_INFO: `/v3/users/3c105a12b12543bf89a681e48e68b469` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:22.968211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c105a12b12543bf89a681e48e68b469)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:22.970161 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:23.015211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c105a12b12543bf89a681e48e68b469'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.022882 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c105a12b12543bf89a681e48e68b469'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.023508 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c105a12b12543bf89a681e48e68b469'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.030570 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c105a12b12543bf89a681e48e68b469'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.031231 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c105a12b12543bf89a681e48e68b469'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.048077 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1303/2595] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:22 2022] DELETE /identity/v3/users/3c105a12b12543bf89a681e48e68b469 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:23.053126 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:23.053528 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:23.053866 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:23.067482 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=100 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:23.074176 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:23.075360 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:23.132720 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1293/2596] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:23.142346 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:23.143838 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:23.157960 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:23.202276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574027cf-08b2-4f1e-8dce-3dab106449c0 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:23.205814 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:23.206700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:23.206700 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:23.207039 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:23.215640 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:23.217053 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:23.227389 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1304/2597] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:23.494300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:23.495867 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:23.507967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8075aa-106e-43d3-b5d3-3d9c94a3f9db None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:23.512559 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:23.513188 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:23.513511 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:23.513787 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] PATH_INFO: `/v3/projects/c20cc2b6d83242faa40568b217cf143d` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:23.514911 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c20cc2b6d83242faa40568b217cf143d)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:23.516826 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:23.564927 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c20cc2b6d83242faa40568b217cf143d', 'request_id': 'req-23689209-30bf-404b-a76e-dd54e3c2ce98'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.572395 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c20cc2b6d83242faa40568b217cf143d', 'request_id': 'req-23689209-30bf-404b-a76e-dd54e3c2ce98'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.579413 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c20cc2b6d83242faa40568b217cf143d', 'request_id': 'req-23689209-30bf-404b-a76e-dd54e3c2ce98'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.590431 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] The token cache is being invalidate because project c20cc2b6d83242faa40568b217cf143d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:23.590718 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.591381 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.592873 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1294/2598] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:23 2022] DELETE /identity/v3/projects/c20cc2b6d83242faa40568b217cf143d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:23.597529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:23.607028 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77271-e3b9-4b56-960f-f2e37cd22b54 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:23.671608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:23.672220 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:23.672496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:23.672771 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] PATH_INFO: `/v3/users/53c1f60ae7ff449ababc87af4c5fb1f7` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:23.673901 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=53c1f60ae7ff449ababc87af4c5fb1f7)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:23.675819 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:23.719578 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53c1f60ae7ff449ababc87af4c5fb1f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.727518 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53c1f60ae7ff449ababc87af4c5fb1f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.728179 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53c1f60ae7ff449ababc87af4c5fb1f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.735046 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53c1f60ae7ff449ababc87af4c5fb1f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.735654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-19c2400f-a000-4afa-a08b-0b5e6d59e9cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53c1f60ae7ff449ababc87af4c5fb1f7'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:23.752949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1305/2599] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:49:23 2022] DELETE /identity/v3/users/53c1f60ae7ff449ababc87af4c5fb1f7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:49:24.019556 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.021119 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:24.058029 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23689209-30bf-404b-a76e-dd54e3c2ce98 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.113081 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.113963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.114245 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.114516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] PATH_INFO: `/v3/projects/5153942312b3478883fa2ef82390ec43` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.115647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5153942312b3478883fa2ef82390ec43)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.117567 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.129057 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:24.165440 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5153942312b3478883fa2ef82390ec43', 'request_id': 'req-65575721-d603-49a6-8461-5f48881374f3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:24.172773 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5153942312b3478883fa2ef82390ec43', 'request_id': 'req-65575721-d603-49a6-8461-5f48881374f3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:24.179513 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5153942312b3478883fa2ef82390ec43', 'request_id': 'req-65575721-d603-49a6-8461-5f48881374f3'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:24.189677 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] The token cache is being invalidate because project 5153942312b3478883fa2ef82390ec43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:49:24.189959 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:24.191151 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-65575721-d603-49a6-8461-5f48881374f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:49:24.192883 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1295/2600] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:49:24 2022] DELETE /identity/v3/projects/5153942312b3478883fa2ef82390ec43 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:24.197772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.198061 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.198335 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.218211 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:24.219409 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:24.239458 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:24.280916 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1306/2601] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 12 12:49:24.289075 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.289397 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.289695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.309985 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:24.311180 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:24.319662 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1296/2602] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:24.323868 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.333833 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347e569-79a6-44d5-ab7f-5fc60925aead None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.338393 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.338987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.339268 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.339539 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.340180 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81424) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 12 12:49:24.340481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.342118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.353581 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1307/2603] 10.222.0.17 () {58 vars in 1153 bytes} [Wed Oct 12 12:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 12 12:49:24.357683 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.367830 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642299af-3b96-4c92-abfb-bf473fca687b None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.372560 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.373157 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.373472 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.373747 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.374379 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.376300 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.403632 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1297/2604] 10.222.0.17 () {60 vars in 1153 bytes} [Wed Oct 12 12:49:24 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 12 12:49:24.407682 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.418267 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e2283a-cecc-4f24-b84a-e285a7a55a6c admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.475134 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.475788 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.476070 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.476344 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] PATH_INFO: `/v3/users` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.476983 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.478949 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.500913 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1308/2605] 10.222.0.17 () {60 vars in 1144 bytes} [Wed Oct 12 12:49:24 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 12 12:49:24.505546 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.515709 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b805c6cc-b2f6-4ff2-ace4-c029444769ff admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.520370 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.520964 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.521242 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.521547 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.522177 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.523774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.533400 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1298/2606] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:24.537242 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.546899 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34420b9d-0856-4a1c-b8a8-db6bbdabf750 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.552110 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef893dce-a47a-4a1d-9579-2a2dcd3de4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.552713 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef893dce-a47a-4a1d-9579-2a2dcd3de4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.552988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef893dce-a47a-4a1d-9579-2a2dcd3de4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.553256 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef893dce-a47a-4a1d-9579-2a2dcd3de4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.553912 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef893dce-a47a-4a1d-9579-2a2dcd3de4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.555496 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef893dce-a47a-4a1d-9579-2a2dcd3de4b8 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.566529 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1309/2607] 10.222.0.17 () {58 vars in 1122 bytes} [Wed Oct 12 12:49:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 12 12:49:24.570405 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.580594 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e8f035-2894-4a58-a510-5b124ea53897 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.585758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.586393 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] REQUEST_METHOD: `PUT` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.586671 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.586943 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] PATH_INFO: `/v3/projects/690179e6164f49bdbfd50103111665de/users/2147319c7cb04f90a2e6bcfb6e10c299/roles/0e327db8aba54600aac0165be40f349b` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.601912 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] RBAC: Authorizing `identity:create_grant(project_id=690179e6164f49bdbfd50103111665de, user_id=2147319c7cb04f90a2e6bcfb6e10c299, role_id=0e327db8aba54600aac0165be40f349b)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.604550 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.617361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1299/2608] 10.222.0.17 () {60 vars in 1483 bytes} [Wed Oct 12 12:49:24 2022] PUT /identity/v3/projects/690179e6164f49bdbfd50103111665de/users/2147319c7cb04f90a2e6bcfb6e10c299/roles/0e327db8aba54600aac0165be40f349b => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:49:24.622562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53859-aa06-49fa-88cf-d76328bde05e None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.623073 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53859-aa06-49fa-88cf-d76328bde05e None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.623199 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e53859-aa06-49fa-88cf-d76328bde05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.641747 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-92e53859-aa06-49fa-88cf-d76328bde05e None None] MFA Rules not processed for user `8feb61a946fd40a291adfe0eb3820253`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:24.642950 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-92e53859-aa06-49fa-88cf-d76328bde05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:24.702478 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1310/2609] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:24.712636 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:24.714346 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:24.782465 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8915e43-181d-4ff8-8fea-07161a319edf admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:24.786251 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:24.786851 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:24.787128 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:24.787398 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:24.796198 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:24.797695 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:24.807000 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1300/2610] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:25.421366 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027fdb-03e9-4af6-82ba-e55f3327e4d4 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:25.421657 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027fdb-03e9-4af6-82ba-e55f3327e4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:25.421930 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027fdb-03e9-4af6-82ba-e55f3327e4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:25.451320 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-65027fdb-03e9-4af6-82ba-e55f3327e4d4 None None] MFA Rules not processed for user `2147319c7cb04f90a2e6bcfb6e10c299`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:25.452446 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-65027fdb-03e9-4af6-82ba-e55f3327e4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:25.474419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:25.522577 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1311/2611] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:25.533299 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:25.535385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:25.568960 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:25.593658 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9abe9b-19e9-47d2-91bd-aea17f2f173a service neutron] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:25.598100 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fde1aa258e9b400983bc8cffa1a8b99c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:25.598687 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:25.598967 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:25.599239 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:25.607758 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:25.609183 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:25.618794 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1301/2612] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:26.981727 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:26.982258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:26.982258 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:27.001904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] MFA Rules not processed for user `2147319c7cb04f90a2e6bcfb6e10c299`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:27.003248 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:27.010154 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1312/2613] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:28.049621 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:28.051034 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:28.142774 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23238c-d537-4c17-9a54-7a74620e8378 service cinder] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:28.148160 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:28.148752 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:28.149032 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:28.149312 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:28.151004 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:28.163282 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:28.164853 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e0023e-2b79-4d9f-8863-ff6709730a84 service placement] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:28.174880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1302/2614] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:36.997132 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=20 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:38.170723 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=53 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:49:42.104027 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:42.105606 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:42.219214 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd481d47-f737-47d3-a7ce-77fd8e48b97b None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:42.225482 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:42.226103 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:42.226384 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:42.226654 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:42.228871 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:42.242777 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:42.244370 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:42.255703 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1313/2615] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:43.462152 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:43.462647 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:43.462762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:43.482155 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] MFA Rules not processed for user `2147319c7cb04f90a2e6bcfb6e10c299`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:49:43.483337 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:43.490961 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1303/2616] 10.222.0.17 () {58 vars in 958 bytes} [Wed Oct 12 12:49:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:49:44.031920 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:44.043761 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ab797-fc73-404a-afbf-45619c2cddaa service nova] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:44.048678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:44.049269 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:44.049630 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:44.049908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:44.051678 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:44.063990 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:44.065642 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d4046d-41f2-45a9-ac62-396d6240f137 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:44.086991 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1314/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:52.588813 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:49:52.590354 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:52.610680 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265f685a-c1fe-4944-abba-d209404abaeb None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:49:52.621610 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:49:52.622224 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:49:52.622499 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:49:52.622772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:49:52.626361 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:49:52.647622 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:49:52.649135 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:49:52.662782 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1304/2618] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:49:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:49:54.071775 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=52 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:02.657193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=9 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:04.689948 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] REQUEST_METHOD: `POST` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:04.690430 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:04.690547 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:04.720419 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.auth.core [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81424) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:04.721625 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:04.774752 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1315/2619] 10.222.0.17 () {58 vars in 1009 bytes} [Wed Oct 12 12:50:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:04.820439 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:04.821850 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:04.888947 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ef2e54-47ae-4fab-9efe-0771761f6877 service nova] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:04.893791 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:04.894385 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:04.894659 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:04.894925 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:04.903174 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:04.904884 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:04.913871 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1305/2620] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:50:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:50:13.780226 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:13.790629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06250343-5c2e-47d8-bf9b-bc8ca7742aa1 None None] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:13.794300 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:13.794904 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:13.795181 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:13.795450 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:13.797301 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:13.868107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:13.869870 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:13.879414 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1316/2621] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 12 12:50:14.908013 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:14.910074 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=32 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:14.942200 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7d7787-0a20-4b08-9ab5-47a74a8dcb43 service swift] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:14.952989 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e2c96c17470485f90e95e3e5bb87ffa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:14.953682 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] REQUEST_METHOD: `GET` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:14.953963 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:14.954234 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:14.956298 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:14.970211 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:14.972544 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aae5367-c6cb-4767-b06d-48b7c66e48d7 service nova] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:14.985408 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1306/2622] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:20.301041 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:20.310709 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d268451-83bd-4dc6-b75d-950a8f759fc7 service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:20.314481 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d0781-7675-4228-a671-247321952abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'afdf63d4c79649a48cddc9f64e10e3e3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:20.315080 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d0781-7675-4228-a671-247321952abf service neutron] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:20.315359 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d0781-7675-4228-a671-247321952abf service neutron] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:20.315629 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d0781-7675-4228-a671-247321952abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:20.317374 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-211d0781-7675-4228-a671-247321952abf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:20.328936 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d0781-7675-4228-a671-247321952abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:20.330441 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d0781-7675-4228-a671-247321952abf service neutron] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:20.340121 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1317/2623] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:21.171604 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] REQUEST_METHOD: `POST` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:21.172103 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:21.172230 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:21.192471 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.auth.core [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] MFA Rules not processed for user `b840fff8907e4931af6a5133eda97e0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81425) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 12 12:50:21.194314 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:21.201448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1307/2624] 10.222.0.17 () {58 vars in 967 bytes} [Wed Oct 12 12:50:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 12 12:50:21.214228 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d0781-7675-4228-a671-247321952abf service neutron] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:21.225887 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d0781-7675-4228-a671-247321952abf service neutron] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:21.231428 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e7f71a2997894b309bd2333a972107d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:21.232107 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:21.232454 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:21.232797 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:21.241854 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:21.243556 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:21.252294 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1318/2625] 10.222.0.17 () {60 vars in 1384 bytes} [Wed Oct 12 12:50:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 12 12:50:23.265262 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:23.281575 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1072123-eb83-4e5c-84b2-2e15cd655940 None None] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:23.287524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:23.288237 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:23.288579 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:23.288917 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] PATH_INFO: `/v3/users/2147319c7cb04f90a2e6bcfb6e10c299` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:23.290137 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2147319c7cb04f90a2e6bcfb6e10c299)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:23.292466 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:23.340186 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2147319c7cb04f90a2e6bcfb6e10c299'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.340892 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2147319c7cb04f90a2e6bcfb6e10c299'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.341531 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2147319c7cb04f90a2e6bcfb6e10c299'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.350880 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2147319c7cb04f90a2e6bcfb6e10c299'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.366219 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2147319c7cb04f90a2e6bcfb6e10c299'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.412327 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1308/2626] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:23 2022] DELETE /identity/v3/users/2147319c7cb04f90a2e6bcfb6e10c299 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:23.581081 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:23.583452 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:23.599249 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13741c-5150-448d-9a0c-496925c73637 service swift] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:23.608196 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:23.610332 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:23.706692 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:23.707313 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:23.707594 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:23.707867 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:23.710376 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:23.725505 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:23.727147 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:23.733561 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=100 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:23.736715 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e4db9-52c0-4c2d-8506-903bd0588b37 admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:23.744549 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:23.745134 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:23.745446 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:23.745895 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] PATH_INFO: `/v3/projects/690179e6164f49bdbfd50103111665de` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:23.747429 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=690179e6164f49bdbfd50103111665de)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:23.752744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1319/2627] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:50:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:50:23.755170 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:23.814524 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '690179e6164f49bdbfd50103111665de', 'request_id': 'req-61e2c561-0147-404d-b094-c30cc2610b5f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.819616 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:23.823425 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '690179e6164f49bdbfd50103111665de', 'request_id': 'req-61e2c561-0147-404d-b094-c30cc2610b5f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.831419 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '690179e6164f49bdbfd50103111665de', 'request_id': 'req-61e2c561-0147-404d-b094-c30cc2610b5f'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.844225 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] The token cache is being invalidate because project 690179e6164f49bdbfd50103111665de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:23.844516 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.845193 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:23.846705 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1309/2628] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:23 2022] DELETE /identity/v3/projects/690179e6164f49bdbfd50103111665de => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 12 12:50:26.926407 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:26.927908 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:26.942790 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cbfba-1b77-4b35-b922-ff6c7bcb9d4a service placement] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.005340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.006772 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.007291 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.007566 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] PATH_INFO: `/v3/users/0401e57284de4fdcace109513142263e` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.008735 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0401e57284de4fdcace109513142263e)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.010704 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.062052 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0401e57284de4fdcace109513142263e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.070943 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0401e57284de4fdcace109513142263e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.071562 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0401e57284de4fdcace109513142263e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.079600 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0401e57284de4fdcace109513142263e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.080277 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0401e57284de4fdcace109513142263e'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.099276 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1320/2629] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:26 2022] DELETE /identity/v3/users/0401e57284de4fdcace109513142263e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:27.281204 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.282772 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:27.306803 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e2c561-0147-404d-b094-c30cc2610b5f admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.367606 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.368164 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.368448 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.368719 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] PATH_INFO: `/v3/projects/1c48992900d140cfbae55f52fd439917` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.369863 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c48992900d140cfbae55f52fd439917)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.371762 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.419266 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c48992900d140cfbae55f52fd439917', 'request_id': 'req-fc637f3d-af06-45a5-aa99-59d950f2ff7e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.426367 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c48992900d140cfbae55f52fd439917', 'request_id': 'req-fc637f3d-af06-45a5-aa99-59d950f2ff7e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.433856 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c48992900d140cfbae55f52fd439917', 'request_id': 'req-fc637f3d-af06-45a5-aa99-59d950f2ff7e'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.445166 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] The token cache is being invalidate because project 1c48992900d140cfbae55f52fd439917 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:27.445481 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.446138 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.447595 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1310/2630] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:27 2022] DELETE /identity/v3/projects/1c48992900d140cfbae55f52fd439917 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:27.451658 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.452988 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:27.465711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a6749c-18be-473e-bd6e-340d92d2f484 admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.521209 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.522064 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.522340 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.522612 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] PATH_INFO: `/v3/users/445f6c7367574b60bc6eed4739d0a07d` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.523719 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] RBAC: Authorizing `identity:delete_user(user_id=445f6c7367574b60bc6eed4739d0a07d)` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.525710 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.545661 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:27.572572 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '445f6c7367574b60bc6eed4739d0a07d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.579744 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '445f6c7367574b60bc6eed4739d0a07d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.580501 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '445f6c7367574b60bc6eed4739d0a07d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.587548 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '445f6c7367574b60bc6eed4739d0a07d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.588170 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.notifications [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '445f6c7367574b60bc6eed4739d0a07d'} {{(pid=81424) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.604892 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1321/2631] 10.222.0.17 () {58 vars in 1224 bytes} [Wed Oct 12 12:50:27 2022] DELETE /identity/v3/users/445f6c7367574b60bc6eed4739d0a07d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:27.776078 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Authenticating user token {{(pid=81425) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:50:27.777784 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.fernet_utils [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81425) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:50:27.793377 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc637f3d-af06-45a5-aa99-59d950f2ff7e admin admin] Validating token access rules against request {{(pid=81425) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:50:27.851864 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8feb61a946fd40a291adfe0eb3820253', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '22af725251c64418b4185f46ec322cf9', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81425) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:50:27.852515 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:50:27.852789 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] SCRIPT_NAME: `/identity` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:50:27.853056 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] PATH_INFO: `/v3/projects/e60af9afb0d842ecb7ce0e99b2bf2ea0` {{(pid=81425) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:50:27.854191 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60af9afb0d842ecb7ce0e99b2bf2ea0)` {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:50:27.856132 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] RBAC: Authorization granted {{(pid=81425) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:50:27.872356 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:27.904295 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60af9afb0d842ecb7ce0e99b2bf2ea0', 'request_id': 'req-7ac53d37-3c43-413a-b354-daf2b906789b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.911626 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60af9afb0d842ecb7ce0e99b2bf2ea0', 'request_id': 'req-7ac53d37-3c43-413a-b354-daf2b906789b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.918679 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60af9afb0d842ecb7ce0e99b2bf2ea0', 'request_id': 'req-7ac53d37-3c43-413a-b354-daf2b906789b'} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.930054 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] The token cache is being invalidate because project e60af9afb0d842ecb7ce0e99b2bf2ea0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81425) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 12 12:50:27.930341 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.930969 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG keystone.notifications [None req-7ac53d37-3c43-413a-b354-daf2b906789b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81425) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 12 12:50:27.932447 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: [pid: 81425|app: 0|req: 1311/2632] 10.222.0.17 () {58 vars in 1233 bytes} [Wed Oct 12 12:50:27 2022] DELETE /identity/v3/projects/e60af9afb0d842ecb7ce0e99b2bf2ea0 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 12 12:50:37.601608 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:DELETE=8,keystone:SELECT=20,keystone:INSERT=1 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:50:37.926387 dv-cinder-846146-8-iscsi devstack@keystone.service[81425]: DEBUG dbcounter [-] [81425] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81425) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 12 12:55:20.542207 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Authenticating user token {{(pid=81424) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 12 12:55:20.543605 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:55:20.611823 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f6d37-7fa2-4ffb-811a-e8bb908a6cce admin admin] Validating token access rules against request {{(pid=81424) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 12 12:55:20.670118 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8740a4be0475453fb422383b9b335d86', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4449657cec624fe4912405ed1757eff1', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81424) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 12 12:55:20.670711 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] REQUEST_METHOD: `GET` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 12 12:55:20.670987 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 12 12:55:20.671251 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81424) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 12 12:55:20.672951 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.fernet_utils [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81424) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 12 12:55:20.683750 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 12 12:55:20.685405 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985a663c-41f4-46c6-bdde-c9664717b0e5 service placement] RBAC: Authorization granted {{(pid=81424) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 12 12:55:20.696092 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: [pid: 81424|app: 0|req: 1322/2633] 10.222.0.17 () {60 vars in 1365 bytes} [Wed Oct 12 12:55:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 12 12:55:30.691492 dv-cinder-846146-8-iscsi devstack@keystone.service[81424]: DEBUG dbcounter [-] [81424] Writing DB stats keystone:SELECT=58 {{(pid=81424) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}